Install Multiple Python Versions in the Image #181
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
It's using a stable version of Debian, so it's redundant to lock packages to specific versions.
Get some of the NsJail build dependencies pre-installed thanks to the base image.
Separate snekbox's Python interpreter from the interpreter used by NsJail. This allows for the interpreters to be updated on different cadences and provides better isolation of packages. Each Python interpreter adds about 70 MB to the built image.
Re-use already built COPY layers in subsequent builds even if the previous layers have changed, which is especially helpful when copying from another build stage. See https://docs.docker.com/engine/reference/builder/#copy---link
Prevent an empty exec_bin.args from manifesting as an empty string in the fully built arguments.
MarkKoz
added
type: feature
Need to use `export` to set vars when && is used between the commands.
|
FWIW, I think this is a good approach as far as the image is concerned. |
MarkKoz
changed the title
ChrisLovering
approved these changes
Aug 26, 2023
jb3
previously requested changes
Aug 29, 2023
Due to the way that `find` executes -exec arguments we need to run the pip install's inside another `sh` instance so that the PYTHONUSERBASE environment variable is correctly picked up. Additionally, we need to specify `--user` so that pip respects the PYTHONUSERBASE variable at all.
jb3
dismissed
their stale review
Dismissing my review since I've added a commit to resolve comments
ChrisLovering
approved these changes
Aug 29, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#158
This is an alternative implementation to #175
This just focuses on getting multiple Python interpreter versions into the image. There will be a follow-up PR with the API changes. The goal for the API is for users to be able to pass an arbitrary path to an executable binary (e.g. a path to a different Python interpreter).
See the changes in .github/CONTRIBUTING.md for an overview of how multiple Python versions are managed.
I observed smaller sizes (~70 MB) on Debian 12 (bookworm), but I've had trouble with test failures on bookworm, so the image remains on Debian 11 (buster) for now. About a third of the size seems to be occupied by the shared library.
If anyone knows of additional compiler flags or configuration options to bring down the size, please share!
Each Python interpreter build takes around 6 minutes, and that's with full optimisation enabled. Thankfully each build stage can run in parallel.