Self signed SSL Certificate support for DevContainers

[jeeftor]

I'm working behind a corporate network that uses SSL inspection and I have thus far been unable to use DevContainers due to SSL issues at work. I'd love the ability to have a trusted cert be able to be inserted into the devcontainers somehow.

Thanks

[egnerfl]

Hi @bamurtaugh ,

This feature is especially important for GitHub Enterprise Customers with own instances.
Hope to see it implemented soon 🎉

Best,
Florian

[bamurtaugh]

Thanks @egnerfl! We'll be sure to provide an update if/when this is added to an iteration plan.

[jeeftor]

I hope its soon!

[robwafle]

Hi, this still seems to be a challenge, roughly a year later. Has this been added to an iteration plan?

[jeeftor]

I'd love to have an update!!!

[fightingsleep]

This would be great since the corporate MITM attack completely prevents the use of dev containers right now

[bdsoha]

@chrmarti @bamurtaugh Any development updates?

A solution to this issue would solve many other (to list a few) open issues:

• Error: self signed certificate in certificate chain #8532
• System Certificates: Support trusted intermediate CAs vscode#177139
• Launching Dev Container - Local workstation w/ corporate MITM SSL erroring vscode-dev-containers#1745
• please expose --no-check-certificate to wslDownload.sh #4444
• Installing VS Code server - Certificate signature failure #5085
• Issue with self signed certificates when installing extensions #2987
• Cloning to development container and extensions fail with self-signed SSL certificate #3713

[jeeftor]

@bdsoha - so it sure seems like it would be worth somebody looking into this :(

[bamurtaugh]

Thank you all for your continued patience and interest in this issue, we really appreciate it.

I've made a note for us to discuss this further during our next month's planning cycle, and you'll be able to see it on an iteration plan if we have a chance to prioritize it (this month's plan as example).

[jeeftor]

@bamurtaugh - I'm at a 10k person company and frankly only a handful of people have implemented dev containers due to the annoyance of our self-signed certs...

[bdsoha]

@jeeftor I find myself in a very similar situation, at a company with 6k employees.
IMO, over time, we see more and more companies that employ stricter firewall, policies, and compliances.
Many of which implement a cooperate CA somewhere in the middle.

[dkwgit]

Same, 5k-10K employee company and this issue prevents good adoption of dev containers. @bamurtaugh. Ty for considering.

[dkwgit]

One additional point: I can make things work if I do a Dockerfile explicitly. (I can copy our certificate into the container, etc.). I can then manually add "features" in the Dockerfile, (i.e. "code explicitly"). But I cannot make things like the "features" section of devcontainer.json work, because that stuff is evaluated in a context where I have not fixed the cert issue. @bamurtaugh.

So the whole features section does not work.

[bdsoha]

@dkwgit I published a feature that is supposed to transparently handle the CA installation into the container during startup.
Due to the abovementioned issues, it does not currently work.

https://github.com/bdsoha/dev-containers

[dkwgit]

Great feature idea @bdsoha. Hope we get to the point where that is doable!!

[shaneholder]

I've been hitting this issue as well. We use a data loss prevention tool that directs all HTTPs traffic through a set of servers that issue self signed certs. I've written up instructions for our company on how to address the issue with lots of different tools but the dev container of VS code is problematic. The way we normally address node programs is to set the environment variable NODE_EXTRA_CA_CERTS to point to the location of the root self signed cert. In fact if I install the devcontainer cli I can run it and have it generate configs for containers. It's like however VS Code is running node it's ignoring that environment variable.

Hope this information helps in troubleshooting.

Oh, and I ran a bunch of different versions, looks like it started around .262.3. Version .255.4 behaves as I would expect.

[dkwgit]

@bamurtaugh Been thinking about this issue a lot. I'm new to dev containers, so there is a lot I don't yet understand and my thoughts may be "off" as a result. But what is impressing itself on me is;

This is not so much about certificate support (that's the symptom here) as it is about a missing lifecycle hook in the devcontainer spec. There is no hook such as initializeCommand that targets the bootstrapping of the devcontainer system (as far as I can tell). If there were such a hook, users could do things like inject their certificates into node at that stage. They could also do other needed things. I realize that the faster (and helpful!) thing to do is provide a way to deal with the certificate issue. (I'd welcome that). But long term, what would really make sense is to have a hook that allows customization at the right moment, imo.

[shaneholder]

@dkwgit Yeah, we ran into that after we build containers and try to install software. We've been copying the root cert into projects and using the Docker build process to copy the cert into the container when it's being built. But the current problem I have is what seems to be a regression in behavior between versions of the extension. I was pretty happy once I discovered I can run the CLI to generate the config, but it's not nearly as nice as using the VSCode UI.

[chrmarti]

How you add a certificate seems to vary by distro (reading https://technotes.shemyak.com/posts/docker-behind-ssl-proxy/).

Features are added after any user-provided Dockerfile runs, so that Dockerfile should be able to add custom certificates.

A feature should be able to add custom certificates too, but since there is no way for the feature to say it wants to run first among any other features, only features that happen to install afterwards benefit from it.

If you have a CI environment without the proxy restriction: You could pre-build the Docker image in CI (devcontainer build) and then use the prebuilt image in a second devcontainer.json locally.

[OneCyrus]

Features are added after any user-provided Dockerfile runs, so that Dockerfile should be able to add custom certificates.

unfortunately this isn‘t the case for more than a year. the features are downloaded outside of this dockerfile after a change in the extension. it worked before.

as far as certificates go. may be just support to import the local trusted certificates into the bootstrap container. this way you don‘t need to support multiple linux distributions.

[chrmarti]

There are multiple points where custom certificates are needed:

• Host OS when running the devcontainer CLI. (NODE_EXTRA_CA_CERTS might work.)
• Bootstrap container for "clone in volume". (Dev Containers extension could copy the OS certificates or NODE_EXTRA_CA_CERTS to the bootstrap container.)
• While building the container image. (Distro-specific.)
• While running the container. (Distro-specific.)

[dkwgit]

Features are added after any user-provided Dockerfile runs, so that Dockerfile should be able to add custom certificates.

@chrmarti In my version of dev container extension, v0.295.0, it does something to resolve features (and fails because of the certificate issue) before it even gets to my Dockerfile. I successfully add the cert in the Dockerfile, so that things inside the container work)--but enabling the features section of my devcontainer.json with any feature immediately causes a failure before the Dockerfile is built. Can post logs if that would help.

[chrmarti]

Just released Dev Containers 0.339.0-pre-release which will check the env variable NODE_EXTRA_CA_CERTS (assuming it is the path to a file with certificates in PEM format) and add the certificates to the bootstrap container that is used for Clone in Volume.

Let me know if that works for you. If it doesn't immediately work, please check that you have indeed set NODE_EXTRA_CA_CERTS (locally when working locally or on the SSH server when going through that) to the path of a file with the required certificates in PEM format.

[OneCyrus]

@chrmarti just saw that there are already libs in vscode which handle the collection of certs from the local cert stores. any reason you don't use that for devcontainers as well?

there's the @vscode/proxy-agent library which uses the @vscode/windows-ca-certs lib on windows to get the certs.

https://github.com/microsoft/vscode-proxy-agent/blob/f8aaf689c36e7add87c9da909736d7ea572d2730/src/index.ts#L576-L593

[chrmarti]

@OneCyrus I will look into using that. (It is not surfaced in the extension API.)

[chrmarti]

Dev Containers 0.340.0-pre-release now loads the same certificates as VS Code and passes these along to the Dev Containers CLI locally and in the bootstrap container when using Clone in Volume. Please give that a try and let me know if it works, thanks!

[OneCyrus]

@chrmarti wow awesome! that's a big step forward. first tests are really promising. I'll test it some more and try to revert all our workarounds to see if something is still missing.

one thing that that would be cool is if we could also consume this cert bundle in our own containers. maybe a devcontainer option to mount this bundle? or is ther already a hidden way to get to this bundle?

[chrmarti]

@chrmarti wow awesome! that's a big step forward. first tests are really promising. I'll test it some more and try to revert all our workarounds to see if something is still missing.

Great, thanks!

one thing that that would be cool is if we could also consume this cert bundle in our own containers. maybe a devcontainer option to mount this bundle? or is ther already a hidden way to get to this bundle?

You could probably pick it up from NODE_EXTRA_CA_CERTS. Though ideally we would find a way to automatically add it, but I expect that to be more involved.

[mholttech]

Can confirm, this worked great in my corporate environment as well!

[OneCyrus]

today i tested also with devcontainer features and that worked as well 👍 the only issue i hit was with the node feature which also is invoking npm which failed with a not trusted self signed certificate error. as far as i can tell npm doesn't use the node settings (would have been too easy ;)). there are a couple of ways to handle this though.

great that we have this now. it will simplify so many things here.

[cymylau]

Any chance someone can share a link on how we elect to use "Dev Containers 0.340.0-pre-release"; my searchfoo is failing me. Thanks.

[mholttech]

Find the extension to install as you normally would and right click on it in the list and select use pre release

[NebraskaCoder]

today i tested also with devcontainer features and that worked as well 👍 the only issue i hit was with the node feature which also is invoking npm which failed with a not trusted self signed certificate error. as far as i can tell npm doesn't use the node settings (would have been too easy ;)). there are a couple of ways to handle this though.

great that we have this now. it will simplify so many things here.

This is what I experienced. I had to comment out the RUN cd && npm i node-pty as it was having CA cert issues. Then everything ran just fine.

Here is the issue:

Text version:

[+] Building 1.9s (10/11)                                docker:rancher-desktop
 => [internal] load build definition from bootstrap.Dockerfile             0.0s
 => => transferring dockerfile: 869B                                       0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for mcr.microsoft.com/devcontainers/base:0-a  0.0s
 => [1/7] FROM mcr.microsoft.com/devcontainers/base:0-alpine-3.16          0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 64.06kB                                       0.0s
 => CACHED [2/7] COPY host-ca-certificates.crt /usr/local/share/ca-certif  0.0s
 => CACHED [3/7] RUN cat /usr/local/share/ca-certificates/host-ca-certifi  0.0s
 => CACHED [4/7] RUN echo "@old https://dl-cdn.alpinelinux.org/alpine/v3.  0.0s
 => CACHED [5/7] RUN apk add --no-cache  git-lfs  nodejs  python3  npm  m  0.0s
 => ERROR [6/7] RUN cd && npm i node-pty                                   1.8s
------
 > [6/7] RUN cd && npm i node-pty:
1.747 npm ERR! code E403
1.749 npm ERR! 403 403 Forbidden - GET https://registry.npmjs.org/node-pty
1.749 npm ERR! 403 In most cases, you or one of your dependencies are requesting
1.749 npm ERR! 403 a package version that is forbidden by your security policy, or
1.750 npm ERR! 403 on a server you do not have access to.
1.751 
1.752 npm ERR! A complete log of this run can be found in:
1.752 npm ERR!     /root/.npm/_logs/2024-02-08T21_13_36_873Z-debug-0.log
------
bootstrap.Dockerfile:23
--------------------
  21 |          ;
  22 |     
  23 | >>> RUN cd && npm i node-pty
  24 |     
  25 |     COPY .vscode-remote-containers /root/.vscode-remote-containers
--------------------
ERROR: failed to solve: process "/bin/sh -c cd && npm i node-pty" did not complete successfully: exit code: 1

Do we have a way to get certs working with NPM in the bootstrap container? Commenting out that run command works fine but I'd rather not have to keep doing that (and have to instruct others in a large company to do that).

[chrmarti]

today i tested also with devcontainer features and that worked as well 👍 the only issue i hit was with the node feature which also is invoking npm which failed with a not trusted self signed certificate error. as far as i can tell npm doesn't use the node settings (would have been too easy ;)). there are a couple of ways to handle this though.

great that we have this now. it will simplify so many things here.

I see NPM can be configured to use the system's CA file (assuming you have your CAs already added there): npm config set cafile /etc/ssl/certs/ca-certificates.crt

@NebraskaCoder I missed that in the bootstrap container, I will change that line to:

RUN npm config set cafile /etc/ssl/certs/ca-certificates.crt && cd && npm i node-pty

Thanks!

[chrmarti]

Dev Containers 0.342.0-pre-release is available with the latest fixes.

[NebraskaCoder]

@NebraskaCoder I missed that in the bootstrap container, I will change that line to:

RUN npm config set cafile /etc/ssl/certs/ca-certificates.crt && cd && npm i node-pty

Thanks!

So, after updating to 0.342.0-pre-release, it still had the same error. I looked into my company's developer "getting started" guide and we block access to the public npm registry. I added RUN npm config set registry="https://npm.company.com/" above your npm command and it worked (I'm sure I could have just added it to the same npm line using an && statement). Are there any options for automating that (maybe grabbing that from the settings)?

[lucaspottersky]

The following still seems to fail when in a corporate MITM certificate scenario. Could this be related to this issue?

.devcontainer:

{
  "name": "Dev Java",
  "image": "mcr.microsoft.com/devcontainers/java:1-21-bullseye",
  "features": {
    "ghcr.io/devcontainers/features/java:1": {
      "installMaven": "true"
    }
  }
}

 > [dev_containers_target_stage 5/5] RUN --mount=type=bind,from=dev_containers_feature_content_source,source=java_0,target=/tmp/build-features-src/java_0     cp -ar /tmp/build-features-src/java_0 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/java_0  && cd /tmp/dev-container-features/java_0  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/java_0:
0.270 ===========================================================================
0.270 Feature       : Java (via SDKMAN!)
0.270 Description   : Installs Java, SDKMAN! (if not installed), and needed dependencies.
0.270 Id            : ghcr.io/devcontainers/features/java
0.270 Version       : 1.4.0
0.270 Documentation : https://github.com/devcontainers/features/tree/main/src/java
0.270 Options       :
0.270     VERSION="latest"
0.270     JDKDISTRO="ms"
0.270     INSTALLGRADLE="false"
0.270     GRADLEVERSION="latest"
0.270     INSTALLMAVEN="true"
0.270     MAVENVERSION="latest"
0.270     INSTALLANT="false"
0.270     ANTVERSION="latest"
0.270     INSTALLGROOVY="false"
0.270     GROOVYVERSION="latest"
0.270 ===========================================================================
0.764 ==== INTERNET NOT REACHABLE! ===================================================
0.764 
0.764  Some functionality is disabled or only partially available.
0.764  If this persists, please enable the offline mode:
0.764 
0.764    $ sdk offline
0.764 
0.764 ================================================================================
0.764 
0.766 This command is not available while offline.
0.767 ERROR: Feature "Java (via SDKMAN!)" (ghcr.io/devcontainers/features/java) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/java for help troubleshooting this error.
------
[2024-02-21T21:04:26.521Z] Dockerfile.extended:26
--------------------
  25 |     ENV PATH="/usr/local/sdkman/bin:/usr/local/sdkman/candidates/java/current/bin:/usr/local/sdkman/candidates/gradle/current/bin:/usr/local/sdkman/candidates/maven/current/bin:/usr/local/sdkman/candidates/ant/current/bin:${PATH}"
  26 | >>> RUN --mount=type=bind,from=dev_containers_feature_content_source,source=java_0,target=/tmp/build-features-src/java_0 \
  27 | >>>     cp -ar /tmp/build-features-src/java_0 /tmp/dev-container-features \
  28 | >>>  && chmod -R 0755 /tmp/dev-container-features/java_0 \
  29 | >>>  && cd /tmp/dev-container-features/java_0 \
  30 | >>>  && chmod +x ./devcontainer-features-install.sh \
  31 | >>>  && ./devcontainer-features-install.sh \
  32 | >>>  && rm -rf /tmp/dev-container-features/java_0
  33 |     
--------------------
[2024-02-21T21:04:26.521Z] ERROR: failed to solve: process "/bin/sh -c cp -ar /tmp/build-features-src/java_0 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/java_0  && cd /tmp/dev-container-features/java_0  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/java_0" did not complete successfully: exit code: 1
[2024-02-21T21:04:26.524Z] Stop (1893 ms): Run: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-s166537/container-features/0.56.0-1708549463216 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/java:1-21-bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-lab-devcontainer-efaaa9b3295ae3fbfefc5dad0b0a7bf3097099c5bc65d7fe74961a54424232cd-features -f /tmp/devcontainercli-s166537/container-features/0.56.0-1708549463216/Dockerfile.extended /home/s166537/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[2024-02-21T21:04:26.524Z] Error: Command failed: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-s166537/container-features/0.56.0-1708549463216 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/java:1-21-bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-lab-devcontainer-efaaa9b3295ae3fbfefc5dad0b0a7bf3097099c5bc65d7fe74961a54424232cd-features -f /tmp/devcontainercli-s166537/container-features/0.56.0-1708549463216/Dockerfile.extended /home/s166537/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[2024-02-21T21:04:26.524Z]     at J$ (/home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:464:1253)
[2024-02-21T21:04:26.524Z]     at $J (/home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:464:997)
[2024-02-21T21:04:26.524Z]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
[2024-02-21T21:04:26.524Z]     at async tAA (/home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:481:3660)
[2024-02-21T21:04:26.524Z]     at async CC (/home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:481:4775)
[2024-02-21T21:04:26.524Z]     at async NeA (/home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:614:11107)
[2024-02-21T21:04:26.524Z]     at async MeA (/home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js:614:10848)
[2024-02-21T21:04:26.528Z] Stop (3566 ms): Run: /usr/share/code/code /home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/s166537/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-5ba7e40b-c406-4520-98ff-2eb2327ee7191708549462294 --workspace-folder /home/s166537/lab-devcontainer --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/s166537/lab-devcontainer --id-label devcontainer.config_file=/home/s166537/lab-devcontainer/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/s166537/lab-devcontainer/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2024-02-21T21:04:26.528Z] Exit code 1
[2024-02-21T21:04:26.529Z] Command failed: /usr/share/code/code /home/s166537/.vscode/extensions/ms-vscode-remote.remote-containers-0.338.1/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/s166537/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-5ba7e40b-c406-4520-98ff-2eb2327ee7191708549462294 --workspace-folder /home/s166537/lab-devcontainer --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/s166537/lab-devcontainer --id-label devcontainer.config_file=/home/s166537/lab-devcontainer/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/s166537/lab-devcontainer/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root
[2024-02-21T21:04:26.529Z] Exit code 1

[chrmarti]

So, after updating to 0.342.0-pre-release, it still had the same error. I looked into my company's developer "getting started" guide and we block access to the public npm registry. I added RUN npm config set registry="https://npm.company.com/" above your npm command and it worked (I'm sure I could have just added it to the same npm line using an && statement). Are there any options for automating that (maybe grabbing that from the settings)?

@NebraskaCoder The NPM extension would be good place for such a setting, unfortunately that request didn't get any traction when it was filed previously: microsoft/vscode#7430. We could use npm config get registry to read the config, but that incurs some runtime costs. Do you have an ~/.npmrc file with that registry configured?

@lucaspottersky Have you configured your proxy for Docker? (https://docs.docker.com/network/proxy/)

[NebraskaCoder]

So, after updating to 0.342.0-pre-release, it still had the same error. I looked into my company's developer "getting started" guide and we block access to the public npm registry. I added RUN npm config set registry="https://npm.company.com/" above your npm command and it worked (I'm sure I could have just added it to the same npm line using an && statement). Are there any options for automating that (maybe grabbing that from the settings)?

@NebraskaCoder The NPM extension would be good place for such a setting, unfortunately that request didn't get any traction when it was filed previously: microsoft/vscode#7430. We could use npm config get registry to read the config, but that incurs some runtime costs. Do you have an ~/.npmrc file with that registry configured?

Yup. Our company has us define a ~/.npmrc file with registry=https://npm.company.com/ as the first line.

[tusv]

@chrmarti Thanks for this fix. It works great. Can I request the same fix for InspectVolume as well?

I tried following to test it manually and it's working.

1. Copied following lines from <random>/vsch/bootstrap-image/0.348.0/bootstrap.Dockerfile to <random>/vsch/inspect-volume/0.348.0/volume.Dockerfile

   COPY host-ca-certificates.crt /tmp/host-ca-certificates.crt
   RUN cat /tmp/host-ca-certificates.crt >> /etc/ssl/certs/ca-certificates.crt
   RUN csplit -f /usr/local/share/ca-certificates/host-ca-certificate- -b '%02d.pem' -z -s /tmp/host-ca-certificates.crt '/-----BEGIN CERTIFICATE-----/' '{*}'
   ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt
   

2. Copied host-ca-certificates.crt from bootstrap-image/0.348.0 to inspect-volume/0.348.0
3. Deleted existing image (vsc-0.348-<random>) for recovery container
4. Opened repository from volume in recovery container

[chrmarti]

@NebraskaCoder node-pty should be optional. I will update the Dockerfile to continue when it can't install it.

@tusv Adding that, thanks.

[chrmarti]

The two mentioned changes are available in Dev Containers 0.356.0-pre-release.

[dal13002]

For some reason, this is still not working for me. When I try to build my dev container using "Rebuild and Reopen container" option, it seems to read my .devcontainer.json file but then I get this error:

------
 > [internal] load metadata for artifactory.my.company.com/go:1-1.22-bullseye:
------
[2024-06-07T02:11:19.861Z] Dockerfile-with-features:3
[2024-06-07T02:11:19.862Z] 
[2024-06-07T02:11:19.862Z] --------------------
[2024-06-07T02:11:19.862Z] 
[2024-06-07T02:11:19.862Z]    1 |     
[2024-06-07T02:11:19.863Z] 
[2024-06-07T02:11:19.864Z]    2 |     	ARG _DEV_CONTAINERS_BASE_IMAGE=placeholder
   3 | >>> FROM  artifactory.my.company.com/go:1-1.22-bullseye AS dev_container_auto_added_stage_label
[2024-06-07T02:11:19.864Z] 
[2024-06-07T02:11:19.864Z]    4 |     
[2024-06-07T02:11:19.864Z] 
[2024-06-07T02:11:19.864Z]    5 |     # Set destination for COPY
[2024-06-07T02:11:19.864Z] 
[2024-06-07T02:11:19.864Z] --------------------
[2024-06-07T02:11:19.865Z] 
[2024-06-07T02:11:19.865Z] ERROR: failed to solve: artifactory.my.company.com/go:1-1.22-bullseye: failed to resolve source metadata for artifactory.my.company.com/go:1-1.22-bullseye: failed to do request: Head "https://artifactory.my.company.com/go/manifests/1-1.22-bullseye": tls: failed to verify certificate: x509: certificate signed by unknown authority
[2024-06-07T02:11:19.872Z] Stop (6338 ms): Run: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /tmp/devcontainercli-dal/container-features/0.62.0-1717726273530/Dockerfile-with-features -t vsc-sample-go-app-97414b8993c589e2a2bc80a45af9955ecde9ba5c289076577a98b7bc00b33560 --target dev_containers_target_stage --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /home/dal/code/sample-go-app/.devcontainer
[2024-06-07T02:11:18.010Z] Error: Command failed: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /tmp/devcontainercli-dal/container-features/0.62.0-1717726273530/Dockerfile-with-features -t vsc-sample-go-app-97414b8993c589e2a2bc80a45af9955ecde9ba5c289076577a98b7bc00b33560 --target dev_containers_target_stage --build-arg _DEV_CONTAINERS_BASE_IMAGE=dev_container_auto_added_stage_label /home/dal/code/sample-go-app/.devcontainer
[2024-06-07T02:11:18.010Z]     at mtA (/home/dal/.vscode-remote-containers/dist/dev-containers-cli-0.369.0/dist/spec-node/devContainersSpecCLI.js:465:1933)
[2024-06-07T02:11:18.010Z]     at async Pm (/home/dal/.vscode-remote-containers/dist/dev-containers-cli-0.369.0/dist/spec-node/devContainersSpecCLI.js:464:1841)
[2024-06-07T02:11:18.011Z]     at async NH (/home/dal/.vscode-remote-containers/dist/dev-containers-cli-0.369.0/dist/spec-node/devContainersSpecCLI.js:464:610)
[2024-06-07T02:11:18.011Z]     at async KtA (/home/dal/.vscode-remote-containers/dist/dev-containers-cli-0.369.0/dist/spec-node/devContainersSpecCLI.js:481:3692)
[2024-06-07T02:11:18.011Z]     at async eB (/home/dal/.vscode-remote-containers/dist/dev-containers-cli-0.369.0/dist/spec-node/devContainersSpecCLI.js:481:4807)
[2024-06-07T02:11:18.011Z]     at async hrA (/home/dal/.vscode-remote-containers/dist/dev-containers-cli-0.369.0/dist/spec-node/devContainersSpecCLI.js:661:13373)
[2024-06-07T02:11:18.011Z]     at async lrA (/home/dal/.vscode-remote-containers/dist/dev-containers-cli-0.369.0/dist/spec-node/devContainersSpecCLI.js:661:13114)

I have my company's private ca certs loaded on the remote host OS, which is bringing up the dev container. It seems like vscode creates a container from moby/buildkit:buildx-stable-1 to help build my image and that container does not have my ca certs. I am using dev containers version v0.369.0
Is there a way to fix this?

[chrmarti]

@dal13002 Not sure where moby/buildkit:buildx-stable-1 comes from, are you using a custom builder? What do you get for running docker buildx ls on the remote host?

[dal13002]

@chrmarti Thank you so much! It seems like I was using a custom builder. I changed the builder back to default and everything works without a problem

[shaneholder]

Hello,

Things are getting better for data loss preventions services that use self signed certs. I am hitting one additional issue though when installing the terraform feature. I'm trying to do this using only the devcontainer.json file.

I am using WSL environment on Windows 10. Below is an example devcontainer.json file I'm using.

// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/ubuntu
{
	"name": "Ubuntu",
	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
	"image": "mcr.microsoft.com/devcontainers/base:jammy",
	"features": {
		"ghcr.io/devcontainers/features/terraform:1": {},
		"ghcr.io/devcontainers/features/azure-cli": {}
	}
}

The issue is that the terraform feature is running curl commands which fail on certificate verification. I'm trying to see if there is a way to get the self signed cert mounted and installed in the container so that curl can see it. It appears that the, onCreateCommand, postCreateCommand and postStartCommand all run after the features are installed. I was attempting to mount the wsl /usr/local/share/ca-certificates folder into the docker image and run update-ca-certificates but it's just not happening early enough it seems. I was able to download the install.sh file for the terraform feature and execute it in a container that performs the above so I'm pretty sure that it will work if I can get the ordering correct.

Thanks,
Shane

[chrmarti]

@shaneholder You need to add the certificates before the features are built. You can do that by using a Dockerfile like, e.g., #6092 (comment). There is also documentation on using the devcontainer.json with a Dockerfile: https://containers.dev/guide/dockerfile . HTH!

[shaneholder]

@chrmarti thanks for that. It's what I ended up doing yesterday. Turns out you can do it with a very minimal Dockerfile and compose.yml file and continue to use the features in devcontainer.json. That last part was an unexpected bonus!