Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore] add multi platform build support #1785

Open
wants to merge 32 commits into
base: main
Choose a base branch
from

Conversation

puckpuck
Copy link
Contributor

@puckpuck puckpuck commented Nov 21, 2024

Changes

Adds 4 new make targets to support building multiplatform images (arm64 & amd64): create-multiplatform-builder, removed-multiplatform-builders, build-multiplatform, and build-multiplatform-and-push.

Dockerfile for nearly all services were updated to correct for warnings emitted by buildx, and to add appropriate build logic for multiple platforms.

Merge Requirements

For new features contributions, please make sure you have completed the following
essential items:

  • CHANGELOG.md updated to document new feature additions
  • Appropriate documentation updates in the docs
  • Appropriate Helm chart updates in the helm-charts

Maintainers will not merge until the above have been completed. If you're unsure
which docs need to be changed ping the
@open-telemetry/demo-approvers.

Makefile Outdated
.PHONY: create-multiplatform-builder
create-multiplatform-builder:
ifeq ($(OS), Darwin)
docker buildx create --name otel-builder --bootstrap --use --driver docker-container --buildkitd-config ./buildkitd.toml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running make create-multiplatform-builder on Mac, I got the following error:

docker buildx create --name otel-builder --bootstrap --use --driver docker-container --buildkitd-config ./buildkitd.toml
unknown flag: --buildkitd-config
See 'docker buildx create --help'.
make: *** [create-multiplatform-builder] Error 125

And when checking --help it seems that the --buildkitd-config doesn't exist:

❯ docker buildx create --help

Usage:  docker buildx create [OPTIONS] [CONTEXT|ENDPOINT]

Create a new builder instance

Options:
      --append                   Append a node to builder instead of changing it
      --bootstrap                Boot builder after creation
      --buildkitd-flags string   Flags for buildkitd daemon
      --config string            BuildKit config file
      --driver string            Driver to use (available: "cloud", "docker-container", "kubernetes", "remote")
      --driver-opt stringArray   Options for the driver
      --leave                    Remove a node from builder instead of changing it
      --name string              Builder instance name
      --node string              Create/modify node with given name
      --platform stringArray     Fixed platforms for current node
      --use                      Set the current builder instance

The command with --config worked fine.

I'm running on M1, macOS 14.7.1, docker version:

docker version
Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.20.10
 Git commit:        afdd53b
 Built:             Thu Oct 26 09:04:20 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.26.1 (131620)
 Engine:
  Version:          master
  API version:      1.44 (minimum version 1.12)
  Go version:       go1.21.3
  Git commit:       54fcd40aa4de94cd75aedc5f6ebf38c6d8f92082
  Built:            Wed Nov 22 07:43:39 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this to use the --config option, which works on MacOs and Linux (I don't have a way to confirm Windows)

linux/amd64 and linux/arm64, use the following command:

```shell
make build-multiplatform
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running this I got the following error:

------
 > [accountingservice linux/arm64 internal] load metadata for mcr.microsoft.com/dotnet/aspnet:8.0:
------
Dockerfile:18
--------------------
  16 |     # -----------------------------------------------------------------------------
  17 |     
  18 | >>> FROM mcr.microsoft.com/dotnet/aspnet:8.0
  19 |     
  20 |     WORKDIR /usr/src/app/
--------------------
ERROR: failed to solve: mcr.microsoft.com/dotnet/aspnet:8.0: failed to resolve source metadata for mcr.microsoft.com/dotnet/aspnet:8.0: unexpected status from HEAD request to https://mcr.microsoft.com/v2/dotnet/aspnet/manifests/8.0: 503 Service Unavailable

View build details: docker-desktop://dashboard/build/otel-builder/otel-builder0/642i8bcqa70fzmljtks1b6tib
make: *** [build-multiplatform] Error 1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a temporary network error while trying to pull the image from the Microsoft registry. Can you try it again?

.PHONY: build-multiplatform-and-push
build-multiplatform-and-push:
# Because buildx bake does not support --env-file yet, we need to load it into the environment first.
set -a; . ./.env.override; set +a && docker buildx bake -f docker-compose.yml --push --set "*.platform=linux/amd64,linux/arm64"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to also create a script that removes the builder.
I've used docker compose build after testing this PR and it tried to use the otel-builder.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a remove-multiplaform-builder target. I also renamed the builder to otel-demo-builder to make it clear why it exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants