feat: Adapt model for RunPod serverless deployment #257
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit introduces changes to allow the model to be deployed on RunPod Serverless. Key changes include: - **`runpod_handler.py`**: A new handler script that serves as the entry point for RunPod. It processes video and audio URL inputs, downloads the files, and then uses the existing `Predictor` class to perform inference. - **`Dockerfile`**: A new Dockerfile to build the container image for RunPod. It includes system dependencies (ffmpeg, libgl1), Python packages from `requirements.txt`, the downloader, and sets the appropriate CMD for the RunPod Python environment. - **`requirements.txt`**: Added the `runpod` package, which is necessary for the RunPod serverless environment. - **`RUNPOD_DEPLOYMENT.md`**: New documentation detailing how to build the Docker image, deploy it to RunPod, and make requests to the serverless endpoint. The existing core inference logic in `predict.py` and `scripts/inference.py` remains largely unchanged, with the new handler acting as an adapter for the RunPod environment and URL-based inputs.
Author
|
update |
The previous Dockerfile was missing `curl`, which is required for downloading the `pget` tool during the image build process. This resulted in a "curl: not found" error. This commit adds `curl` to the `apt-get install` command in the Dockerfile to ensure it is available.
The Docker build was failing during `pip install` because the `insightface` package could not be built from source. This typically occurs when system-level build tools and development libraries are missing. This commit updates the Dockerfile to install `build-essential`, `python3-dev`, and `cmake` via `apt-get`. These packages provide the necessary C/C++ compilers, Python development headers, and build system tools required for compiling `insightface` and other similar Python packages with native extensions.
|
I ran a test and found that runpod/serverless is giving an error: 'worker exited with exit code 0'. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit introduces changes to allow the model to be deployed on RunPod Serverless.
Key changes include:
runpod_handler.py: A new handler script that serves as the entry point for RunPod. It processes video and audio URL inputs, downloads the files, and then uses the existingPredictorclass to perform inference.Dockerfile: A new Dockerfile to build the container image for RunPod. It includes system dependencies (ffmpeg, libgl1), Python packages fromrequirements.txt, the downloader, and sets the appropriate CMD for the RunPod Python environment.requirements.txt: Added therunpodpackage, which is necessary for the RunPod serverless environment.RUNPOD_DEPLOYMENT.md: New documentation detailing how to build the Docker image, deploy it to RunPod, and make requests to the serverless endpoint.The existing core inference logic in
predict.pyandscripts/inference.pyremains largely unchanged, with the new handler acting as an adapter for the RunPod environment and URL-based inputs.