Add a guide to build custom Beam Python SDK image #33048
Conversation
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
baeminbo
force-pushed
the
beam-python-sdk-image
branch
from
145821e
to
e6a14f2
Compare
baeminbo
force-pushed
the
beam-python-sdk-image
branch
from
e6a14f2
to
19be319
Compare
|
R: @tvalentyn |
|
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment |
| # Create a simple Dockerfile to use custom boot entrypoint. | ||
| cat >Dockerfile <<EOF | ||
| FROM apache/beam_python3.10_sdk:2.60.0 | ||
| COPY boot /opt/apache/beam/boot_modified |
There was a problem hiding this comment.
will docker overwrite the file if we don't change the name? we could keep it as is to avoid having two entrypoints.
|
|
||
| ## (Optional) Update Boot Application Only. | ||
|
|
||
| If you only need to make a change to the Python SDK boot application. You can |
There was a problem hiding this comment.
| If you only need to make a change to the Python SDK boot application. You can | |
| If you only need to make a change to the Python SDK boot entrypoint binary, you can |
| docker push <my-account>/beam_python3.10_sdk:2.60.0-custom | ||
| ``` | ||
|
|
||
| ## (Optional) Update Boot Application Only. |
There was a problem hiding this comment.
I think we should start with this, in case this is sufficient, as steps are simpler.
| # Building Beam Python SDK Image Guide | ||
|
|
||
| You can build a docker image if your local environment has Java, Python, Golang | ||
| and Docker properly. Try |
There was a problem hiding this comment.
| and Docker properly. Try | |
| and Docker installation. Try |
| @@ -44,6 +44,7 @@ | |||
| <li><a href="/documentation/sdks/python-pipeline-dependencies/">Managing pipeline dependencies</a></li> | |||
| <li><a href="/documentation/sdks/python-multi-language-pipelines/">Python multi-language pipelines quickstart</a></li> | |||
| <li><a href="/documentation/sdks/python-unrecoverable-errors/">Python Unrecoverable Errors</a></li> | |||
| <li><a href="/documentation/sdks/python-sdk-image-build/">Python SDK image build</a></li> | |||
There was a problem hiding this comment.
We have related content in: https://beam.apache.org/documentation/runtime/environments/. I think it might be easier to find this link if we add a reference from that page.
We could also integrate (a portion of?) this this content into that page, but having only have a reference : for self-contained instructions on building a container image, follow this guide (link to new page), is ok too.
|
Thanks a lot for putting this together! |
This is a guide for users who needs customized Python SDK image. b/375969331
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.