-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Prettified Code! fix axum rework move custom runtimes to Containers
- Loading branch information
1 parent
3f55543
commit 0edefab
Showing
3 changed files
with
170 additions
and
14 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,136 @@ | ||
| --- | ||
| title: "Containerized Runtimes with Custom Dockerfiles" | ||
| description: "Run generic containerized applications on Cerebrium with your own Dockerfile" | ||
| --- | ||
|
|
||
| For even greater flexibility, you are able to bring your own, already functional containerized applications to Cerebrium! You can build anything from standard Python to | ||
| compiled Rust, as long as you supply a `Dockerfile` to build your Cerebrium app with. | ||
|
|
||
| ## Building Dockerized Python applications | ||
|
|
||
| Suppose we have a simple FastAPI server we have already dockerized. | ||
|
|
||
| ```python | ||
| from fastapi import FastAPI | ||
|
|
||
| app = FastAPI() | ||
|
|
||
| @app.post("/hello") | ||
| def hello(): | ||
| return {"message": "Hello Cerebrium!"} | ||
|
|
||
| @app.get("/health") | ||
| def health(): | ||
| return "Ok" | ||
| ``` | ||
|
|
||
| Since this application is dockerized, we have a simple, previously defined `Dockerfile`: | ||
|
|
||
| ```dockerfile | ||
| FROM python:3.12-bookworm | ||
| COPY . . | ||
| RUN pip install uvicorn fastapi | ||
| EXPOSE 5000 | ||
| CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "5000"] | ||
| ``` | ||
|
|
||
| This is just a regular Dockerfile. | ||
| However, it does include a `CMD` clause so that the container can be run. | ||
| We can leverage this Dockerfile in Cerebrium by using it directly to build this application. | ||
|
|
||
| Configure this application in `cerebrium.toml` by adding a custom runtime section with a `dockerfile_path` parameter: | ||
|
|
||
| ```toml | ||
| [cerebrium.runtime.custom] | ||
| port = 5000 | ||
| healthcheck_endpoint = "/health" | ||
| dockerfile_path = "./Dockerfile" | ||
| ``` | ||
|
|
||
| The configuration requires three key parameters: | ||
|
|
||
| - `port`: The port your server listens on | ||
| - `healthcheck_endpoint`: The endpoint that confirms server health. If empty, will default to a TCP ping of your specified port | ||
| - `dockerfile_path`: The relative path of the project Dockerfile used to build the application | ||
|
|
||
| If your Dockerfile does not contain a `CMD` clause, you should also specify `entrypoint` in your `cerebrium.toml`: | ||
|
|
||
| ```toml | ||
| [cerebrium.runtime.custom] | ||
| entrypoint = ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "5000"] | ||
| ... | ||
| ``` | ||
|
|
||
| <Info> | ||
| If a dockerfile path is specified, you must take care to install all the | ||
| dependencies your app needs in the Dockerfile, as well as run any commands. | ||
| All dependencies specified under `cerebrium.dependencies.*` will be ignored, | ||
| and `cerebrium.deployment.shell_commands` & | ||
| `cerebrium.deployment.pre_build_commands` will not be run. | ||
| </Info> | ||
|
|
||
| ## Building Generic Dockerized applications | ||
|
|
||
| So long as you supply a Dockerfile, you are not limited to just Python! | ||
| Here's an equivalent Rust based API server in the Axum Framework to demonstrate how this works: | ||
|
|
||
| ```rust | ||
| use axum::{ | ||
| routing::{get, post}, | ||
| Json, Router, | ||
| }; | ||
| use serde_json::json; | ||
| use std::net::SocketAddr; | ||
| use tracing::Level; | ||
|
|
||
| async fn hello() -> Json<serde_json::Value> { | ||
| Json(json!({ "message": "Hello Cerebrium!" })) | ||
| } | ||
|
|
||
| async fn health() -> &'static str { | ||
| "Ok" | ||
| } | ||
|
|
||
| #[tokio::main] | ||
| async fn main() { | ||
| let app = Router::new() | ||
| .route("/hello", post(hello)) | ||
| .route("/health", get(health)); | ||
| let addr = SocketAddr::from(([127, 0, 0, 1], 5000)); | ||
| tracing::info!("Listening on {}", addr); | ||
| axum::Server::bind(&addr.parse().unwrap()) | ||
| .serve(app.into_make_service()) | ||
| .await | ||
| .unwrap(); | ||
| } | ||
| ``` | ||
|
|
||
| In this case, we use a multi-stage Dockerfile: | ||
|
|
||
| ```dockerfile | ||
| # Build Stage | ||
| FROM rust:bookworm as builder | ||
| WORKDIR /usr/src/app | ||
| COPY Cargo.toml Cargo.lock ./ | ||
| RUN cargo fetch | ||
|
|
||
| # Copy the rest of the application source code and Build | ||
| COPY . . | ||
| RUN cargo build --release | ||
|
|
||
|
|
||
| # Runtime Stage | ||
| FROM scratch | ||
| COPY --from=builder /usr/src/app/target/release/axum_server /app | ||
| EXPOSE 5000 | ||
| CMD ["/app"] | ||
| ``` | ||
|
|
||
| Configure this application in `cerebrium.toml` similarly to the FastAPI webserver and we are done! | ||
|
|
||
| ```toml | ||
| [cerebrium.runtime.custom] | ||
| port = 5000 | ||
| healthcheck_endpoint = "/health" | ||
| dockerfile = "./Dockerfile" | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters