Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue building arm64 Credentials Docker image #2477

Closed
justinhynes opened this issue May 9, 2024 · 2 comments
Closed

Issue building arm64 Credentials Docker image #2477

justinhynes opened this issue May 9, 2024 · 2 comments
Labels
maintenance Routine upkeep necessary for the health of the platform needs maintainer attention Issue or PR specifically needs the attention of the maintainer.

Comments

@justinhynes
Copy link
Contributor

On 2024-05-08 an update was made to the public Dockerfile to support Python 3.11 (#2474).

After this merged to the master branch, I noticed that the Build and Push Docker Images workflow has started to fail.

Before merging my changes, I had verified a successful build of a docker image from the repo's base Dockerfile. Upon further inspection, I realized that we are building multiple versions of the image for multiple platforms (linux/amd64 & linux/arm64, configured here -- https://github.com/openedx/credentials/actions/runs/9014850121/workflow#L53).

This issue seems specifically related to the image build for linux/arm64. It is having an issue finding a compatible version of the didkit dependency for the arm64 image.

Logs from a failed workflow: https://github.com/openedx/credentials/actions/runs/9014850121/job/24768334656.

We first build the linux/amd64 version, which is able to locate and install didkit v. 0.3.2:

#26 4.081 Collecting didkit==0.3.2 (from -r requirements/production.txt (line 77))
#26 4.103   Downloading didkit-0.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.8 kB)
...
#26 15.47 Downloading didkit-0.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.6 MB)
#26 15.58    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.6/10.6 MB 95.0 MB/s eta 0:00:00
...
Installing collected packages: webencodings, text-unidecode, pytz, python-memcached, pypng, polib, openedx-atlas, itypes, fontawesomefree, django-webpack-loader, django-sortedm2m, zope-interface, zope-event, urllib3, uritemplate, typing-extensions, sqlparse, six, simplejson, semantic-version, pyyaml, python-slugify, pymemcache, pyjwt, pygments, pycparser, psutil, pillow, pbr, path, oauthlib, newrelic, mysqlclient, markupsafe, markdown, lxml, jmespath, inflection, idna, gunicorn, greenlet, fastavro, dnspython, django-ratelimit, didkit, ...
...
Successfully installed asgiref-3.8.1 attrs-23.2.0 backoff-2.2.1 bleach-6.1.0 boto3-1.34.99 botocore-1.34.99 certifi-2024.2.2 cffi-1.16.0 charset-normalizer-3.3.2 code-annotations-1.8.0 coreapi-2.3.3 coreschema-0.0.4 cryptography-42.0.7 defusedxml-0.8.0rc2 didkit-0.3.2...

However, during the image build for linux/arm64, we run into issues:

#50 21.22 ERROR: Could not find a version that satisfies the requirement didkit==0.3.2 (from versions: 0.0.1, 0.2.1)
#50 21.22 ERROR: No matching distribution found for didkit==0.3.2
...
21.22 ERROR: Could not find a version that satisfies the requirement didkit==0.3.2 (from versions: 0.0.1, 0.2.1)
21.22 ERROR: No matching distribution found for didkit==0.3.2
------
Dockerfile:67
--------------------
  65 |     # Dependencies are installed as root so they cannot be modified by the application user.
  66 |     RUN pip install -r requirements/pip_tools.txt
  67 | >>> RUN pip install -r requirements/production.txt
  68 |     
  69 |     RUN mkdir -p /edx/var/log
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements/production.txt" did not complete successfully: exit code: 1

I'm not sure if this is caused by an incompatibility between Python 3.11 and didkit. I have not rolled back the changes to the public Dockerfile as Python 3.11 was a requirement for the Redwood release of Open edX.

@justinhynes justinhynes changed the title Issue building arm64 Credentials image Issue building arm64 Credentials Docker image May 9, 2024
@justinhynes
Copy link
Contributor Author

justinhynes commented May 9, 2024

We recently asked AXIM around the use of the public Dockerfile in the Credentials repo and were told that we (2U) were the only folks using it.

We are going to update the Build and Push Docker Images workflow to not build the arm64 version for now.

justinhynes added a commit that referenced this issue May 9, 2024
There seems to be an issue installing a required dependency (`didkit`) for the `arm64` platform. Since one of the two platforms are failing, it causes the good image for the `linux/amd64` platform to not upload either.

This is keeping us from being able to push an updated Docker image running Python 3.11 to DockerHub and impeding Devstack developers.

We are under the impression that only 2U uses this Dockerfile and this shouldn't affect the larger community.

For additional details, please see the public Credentials issue: #2477.
justinhynes added a commit that referenced this issue May 9, 2024
There seems to be an issue installing a required dependency (`didkit`) for the `arm64` platform. Since one of the two platforms are failing, it causes the good image for the `linux/amd64` platform to not upload either.

This is keeping us from being able to push an updated Docker image running Python 3.11 to DockerHub and impeding Devstack developers.

We are under the impression that only 2U uses this Dockerfile and this shouldn't affect the larger community.

For additional details, please see the public Credentials issue: #2477.
@deborahgu deborahgu moved this to Backlog in Aperture-Maintained Jul 12, 2024
@deborahgu deborahgu added maintenance Routine upkeep necessary for the health of the platform needs maintainer attention Issue or PR specifically needs the attention of the maintainer. labels Jul 12, 2024
@deborahgu deborahgu moved this from Backlog to Triage in Aperture-Maintained Jul 15, 2024
@deborahgu deborahgu moved this from Triage to Backlog in Aperture-Maintained Jul 17, 2024
@justinhynes
Copy link
Contributor Author

This issue has been resolved and ARM64 builds have been reenabled. Thanks for the assist here working with didkit's maintainers @GlugovGrGlib!

@github-project-automation github-project-automation bot moved this from Backlog to Done in Aperture-Maintained Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Routine upkeep necessary for the health of the platform needs maintainer attention Issue or PR specifically needs the attention of the maintainer.
Projects
Status: Done
Development

No branches or pull requests

2 participants