Running aarch64 codebuild image on a x86_64 machine

[DavidHospital]

I have a Lambda function with arm64 architecture running Python 3.9, which is deployed to our production environment using CodePipeline using the amazonlinux2-aarch64-standard:2.0 image. This all works fine. By using this image, aarch64 versions of the python dependencies for the Lambda can be installed using buildspec.yml and tools like pytest can run in an aarch64 environment during the build process.

The problem arises when building and deploying a copy of the lambda for development purposes by building locally using the codebuild_build.sh script. Since my machine has x86_64 architecture, this fails (logs below).

Describe the issue
Unable to run public.ecr.aws/codebuild/amazonlinux2-x86_64-standard:4.0 codebuild image using codebuild_build.sh on an x86_64 machine.

To Reproduce
Steps to reproduce the behavior:

1. pull amazonlinux2-aarch64-standard:2.0 docker image on an x86_64 machine

docker pull public.ecr.aws/codebuild/amazonlinux2-aarch64-standard:2.0

2. run ./build_scripts/local/codebuild_build.sh -i aws/codebuild/amazonlinux2-aarch64-standard:2.0 -a tmp/

Expected behavior
Build should run without issue.

Logs

Build Command:

docker run -it -v /var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=aws/codebuild/amazonlinux2-aarch64-standard:2.0" -e "ARTIFACTS=/home/dhospital/work/feature-flags-service/tmp/" -e "SOURCE=/home/dhospital/work/feature-flags-service" -e "INITIATOR=dhospital" public.ecr.aws/codebuild/local-builds:latest

Removing agent-resources_build_1 ... done
Removing agent-resources_agent_1 ... done
Removing network agent-resources_default
Removing volume agent-resources_source_volume
Removing volume agent-resources_user_volume
Creating network "agent-resources_default" with the default driver
Creating volume "agent-resources_source_volume" with local driver
Creating volume "agent-resources_user_volume" with local driver
Creating agent-resources_agent_1 ... done
Creating agent-resources_build_1 ... done
Attaching to agent-resources_agent_1, agent-resources_build_1
build_1  | exec /usr/bin/sh: exec format error
agent-resources_build_1 exited with code 1
Stopping agent-resources_agent_1 ... done
Aborting on container exit...

Platform (please complete the following information):

• OS: Linux x86

[Rogalek]

I have the same issue, please help!

[srgoni]

If you're working on a Linux system, there is a way to run cross-architecture containers - user-mode QEMU is your friend.

Instructions for Debian+Podman:

apt install qemu-user-static podman
# might need to reboot here, to enable qemu's binfmt_misc hooks
podman run -ti --rm --arch aarch64 public.ecr.aws/codebuild/amazonlinux2-aarch64-standard:2.0

This will treat the container like any other, but now the kernel will recognize that the executables are for a different CPU architecture and automatically execute them with the corresponding qemu emulator. System calls are translated and handed off to the native kernel, so the performance impact is actually not so bad in many cases.

Should also work in a similar fashion on RPM-based distributions and/or Docker (but I haven't tested it).