vscode-remote-release
vscode-remote-release copied to clipboard
microsoft
Self signed SSL Certificate support for DevContainers
I'm working behind a corporate network that uses SSL inspection and I have thus far been unable to use DevContainers due to SSL issues at work. I'd love the ability to have a trusted cert be able to be inserted into the devcontainers somehow.
Thanks
This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 10 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!
:slightly_smiling_face: This feature request received a sufficient number of community upvotes and we moved it to our backlog. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!
Hi @bamurtaugh ,
This feature is especially important for GitHub Enterprise Customers with own instances. Hope to see it implemented soon 🎉
Best, Florian
Thanks @egnerfl! We'll be sure to provide an update if/when this is added to an iteration plan.
Hi, this still seems to be a challenge, roughly a year later. Has this been added to an iteration plan?
This would be great since the corporate MITM attack completely prevents the use of dev containers right now
@chrmarti @bamurtaugh Any development updates?
A solution to this issue would solve many other (to list a few) open issues:
- https://github.com/microsoft/vscode-remote-release/issues/8532
- https://github.com/microsoft/vscode/issues/177139
- https://github.com/microsoft/vscode-dev-containers/issues/1745
- https://github.com/microsoft/vscode-remote-release/issues/4444
- https://github.com/microsoft/vscode-remote-release/issues/5085
- https://github.com/microsoft/vscode-remote-release/issues/2987
- https://github.com/microsoft/vscode-remote-release/issues/3713
@bdsoha - so it sure seems like it would be worth somebody looking into this :(
Thank you all for your continued patience and interest in this issue, we really appreciate it.
I've made a note for us to discuss this further during our next month's planning cycle, and you'll be able to see it on an iteration plan if we have a chance to prioritize it (this month's plan as example).
@bamurtaugh - I'm at a 10k person company and frankly only a handful of people have implemented dev containers due to the annoyance of our self-signed certs...
@jeeftor I find myself in a very similar situation, at a company with 6k employees. IMO, over time, we see more and more companies that employ stricter firewall, policies, and compliances. Many of which implement a cooperate CA somewhere in the middle.
Same, 5k-10K employee company and this issue prevents good adoption of dev containers. @bamurtaugh. Ty for considering.
One additional point: I can make things work if I do a Dockerfile explicitly. (I can copy our certificate into the container, etc.). I can then manually add "features" in the Dockerfile, (i.e. "code explicitly"). But I cannot make things like the "features" section of devcontainer.json work, because that stuff is evaluated in a context where I have not fixed the cert issue. @bamurtaugh.
So the whole features section does not work.
@dkwgit I published a feature that is supposed to transparently handle the CA installation into the container during startup. Due to the abovementioned issues, it does not currently work.
https://github.com/bdsoha/dev-containers
I've been hitting this issue as well. We use a data loss prevention tool that directs all HTTPs traffic through a set of servers that issue self signed certs. I've written up instructions for our company on how to address the issue with lots of different tools but the dev container of VS code is problematic. The way we normally address node programs is to set the environment variable NODE_EXTRA_CA_CERTS to point to the location of the root self signed cert. In fact if I install the devcontainer cli I can run it and have it generate configs for containers. It's like however VS Code is running node it's ignoring that environment variable.
Hope this information helps in troubleshooting.
Oh, and I ran a bunch of different versions, looks like it started around .262.3. Version .255.4 behaves as I would expect.
@bamurtaugh Been thinking about this issue a lot. I'm new to dev containers, so there is a lot I don't yet understand and my thoughts may be "off" as a result. But what is impressing itself on me is;
This is not so much about certificate support (that's the symptom here) as it is about a missing lifecycle hook in the devcontainer spec. There is no hook such as initializeCommand that targets the bootstrapping of the devcontainer system (as far as I can tell). If there were such a hook, users could do things like inject their certificates into node at that stage. They could also do other needed things. I realize that the faster (and helpful!) thing to do is provide a way to deal with the certificate issue. (I'd welcome that). But long term, what would really make sense is to have a hook that allows customization at the right moment, imo.
@dkwgit Yeah, we ran into that after we build containers and try to install software. We've been copying the root cert into projects and using the Docker build process to copy the cert into the container when it's being built. But the current problem I have is what seems to be a regression in behavior between versions of the extension. I was pretty happy once I discovered I can run the CLI to generate the config, but it's not nearly as nice as using the VSCode UI.
How you add a certificate seems to vary by distro (reading https://technotes.shemyak.com/posts/docker-behind-ssl-proxy/).
Features are added after any user-provided Dockerfile runs, so that Dockerfile should be able to add custom certificates.
A feature should be able to add custom certificates too, but since there is no way for the feature to say it wants to run first among any other features, only features that happen to install afterwards benefit from it.
If you have a CI environment without the proxy restriction: You could pre-build the Docker image in CI (devcontainer build) and then use the prebuilt image in a second devcontainer.json locally.
Features are added after any user-provided Dockerfile runs, so that Dockerfile should be able to add custom certificates.
unfortunately this isn‘t the case for more than a year. the features are downloaded outside of this dockerfile after a change in the extension. it worked before.
as far as certificates go. may be just support to import the local trusted certificates into the bootstrap container. this way you don‘t need to support multiple linux distributions.
There are multiple points where custom certificates are needed:
- Host OS when running the devcontainer CLI. (NODE_EXTRA_CA_CERTS might work.)
- Bootstrap container for "clone in volume". (Dev Containers extension could copy the OS certificates or NODE_EXTRA_CA_CERTS to the bootstrap container.)
- While building the container image. (Distro-specific.)
- While running the container. (Distro-specific.)
Features are added after any user-provided Dockerfile runs, so that Dockerfile should be able to add custom certificates.
@chrmarti In my version of dev container extension, v0.295.0, it does something to resolve features (and fails because of the certificate issue) before it even gets to my Dockerfile. I successfully add the cert in the Dockerfile, so that things inside the container work)--but enabling the features section of my devcontainer.json with any feature immediately causes a failure before the Dockerfile is built. Can post logs if that would help.
@chrmarti Like previously mentioned in the comment above, I created a dedicated feature, that is meant to install the CA cert in the running container.
Like @dkwgit mentioned, the process fails before any attempts to build an image begin.
With me it looks easier:
I just need the feature to add the certificates to the bootstrap (and the inspect-volume) container - since those are pulling in an image from alpine and then directly try to download the rest. This is the most annoying part, since this means I always have to patch the VSC-Dockerfiles before my containers work again...
The dev-container i can put my own container as the basis, where I already install the ca-cert. So that is in my hands, there is at least a "stable" solution around.
@dkwgit @bdsoha The features are downloaded on the host machine. You could use the NODE_EXTRA_CA_CERTS environment variable to work around the missing support (bash syntax below):
NODE_EXTRA_CA_CERTS=/path/to/cert.pem devcontainer build --workspace-folder .
The features are downloaded on the host machine. You could use the NODE_EXTRA_CA_CERTS environment variable to work around the missing support (bash syntax below):
NODE_EXTRA_CA_CERTS=/path/to/cert.pem devcontainer build --workspace-folder .
That worked for me. I had tried that before, but must have had something wrong (maybe the cert path). THANK YOU! This definitely is an improvement for me.
@chrmarti: So I am able to build with the dev container cli, when I have correctly set NODE_EXTRA_CA_CERTS. Oddly, however, the only way to work within the dev container from the GUI appears to be an attach to the running container. Cannot just open the folder in it. It tries to re-evaluate the features and that fails, even though I have NODE_EXTRA_CA_CERTS set the same way that works with the dev container cli.
I think part of the problem here for me (and most corporate devs) is Windows. I suspect that a lot of the extension developers are not on Windows (or don't have corporate MITM certs involved) and so it all just works pretty well. But Node's non existent cert store connectedness in Windows (this has been going on for years) just makes the whole stack fragile in the Windows/corporate space.
I think what I was doing wrong before was setting NODE_EXTRA_CA_CERTS to an absolute windows path. That did not work. But if I set it to a path correctly relative to the devcontainer.json file, it works via the devcontainer cli.
When running it with the Dev Containers extension in VS Code we hit https://github.com/electron/electron/issues/10257. I can add code to load the NODE_EXTRA_CA_CERTS file in this case.
The above mentioned fix for NODE_EXTRA_CA_CERTS is available with Dev Containers 0.297.0-pre-release.
Unsure if the above mentioned fix was intended to address the initial problem reported, but I just tried 0.298.0 and I'm still having problems, this is on Win 10 with WSL2 installed.
Version: 1.79.2 (system setup) Commit: 695af097c7bd098fbf017ce3ac85e09bbc5dda06 Date: 2023-06-14T08:57:04.379Z Electron: 22.5.7 Chromium: 108.0.5359.215 Node.js: 16.17.1 V8: 10.8.168.25-electron.0 OS: Windows_NT x64 10.0.19044
Simple Alpine container.
It looks like there are logs before the below, but VSCode restarts too quickly for me to capture them.
[24 ms] Dev Containers 0.298.0 in VS Code 1.79.2 (695af097c7bd098fbf017ce3ac85e09bbc5dda06).
[24 ms] Start: Resolving Remote
[34 ms] Start: Run: wsl -l -v
[665 ms] Start: Run: wsl -d Ubuntu-18.04 -e /bin/sh -c echo ~
[1586 ms] Start: Run: wsl -d Ubuntu-18.04 -e /bin/sh -c cd '/home/<MYUSER>' && /bin/sh
[1664 ms] Start: Run in host: id -un
[1981 ms] <MYUSER>
[1981 ms]
[1982 ms] Start: Run in host: cat /etc/passwd
[1983 ms] Start: Run in host: echo ~
[1984 ms] /home/<MYUSER>
[1984 ms]
[1985 ms] Start: Run in host: test -x '/home/<MYUSER>/.vscode-remote-containers/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06/node'
[1986 ms]
[1986 ms]
[1986 ms] Exit code 1
[1987 ms] Start: Run in host: test -x '/home/<MYUSER>/.vscode-server/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06/node'
[1988 ms]
[1988 ms]
[1988 ms] Exit code 1
[1988 ms] Start: Run in host: uname -s
[1989 ms] Linux
[1989 ms]
[1989 ms] Start: Run in host: uname -m
[1990 ms] x86_64
[1991 ms]
[1991 ms] Start: Run in host: (cat /etc/os-release || cat /usr/lib/os-release) 2>/dev/null
[1992 ms] NAME="Ubuntu"
VERSION="18.04.6 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.6 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
[1992 ms]
[1993 ms] Start: Downloading VS Code Server
[1993 ms] 695af097c7bd098fbf017ce3ac85e09bbc5dda06 linux-x64 stable
[3770 ms] Start: Installing VS Code Server
[3770 ms] Start: Run in host: mkdir -p /home/<MYUSER>/.vscode-remote-containers/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06_1688151804857
[3773 ms]
[3774 ms]
[3775 ms] Start: Run in host: (dd iflag=fullblock bs=8192 count=10051 2>/dev/null; dd iflag=fullblock bs=3072 count=1 2>/dev/null) | tar --no-same-owner -x -C /home/<MYUSER>/.vscode-remote-containers/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06_1688151804857
[3934 ms]
[3935 ms]
[3935 ms] Start: Run in host: mv -n /home/<MYUSER>/.vscode-remote-containers/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06_1688151804857/* /home/<MYUSER>/.vscode-remote-containers/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06 && rmdir /home/<MYUSER>/.vscode-remote-containers/bin/695af097c7bd098fbf017ce3ac85e09bbc5dda06_1688151804857
[3939 ms]
[3939 ms]
[3940 ms] Start: Run in host: test -f '/home/<MYUSER>/.vscode-remote-containers/dist/vscode-remote-containers-server-0.298.0.js'
[3941 ms]
[3941 ms]
[3941 ms] Exit code 1
[3942 ms] Start: Run in host: mkdir -p '/home/<MYUSER>/.vscode-remote-containers/dist'
[3944 ms]
[3944 ms]
[3944 ms] Start: Run in host: cat <<'EOF-/home/<MYUSER>/.vscode-remote-containers/dist/vscode-remote-containers-server-0.298.0.js' >/home/<MYUSER>/.vscode-remote-containers/dist/vscode-remote-containers-server-0.298.0.js_1688151805031
[3947 ms]
[3947 ms]
[3949 ms] userEnvProbe: loginInteractiveShell (default)
[3949 ms] userEnvProbe: not found in cache
[3950 ms] userEnvProbe shell: /bin/bash
[4115 ms] userEnvProbe PATHs:
Probe: '<REMOVED>'
Container: None
[4127 ms] Start: Run in Host: wslpath -w /mnt/wslg/runtime-dir/wayland-0
[4133 ms] Start: Check Docker is running
[4133 ms] Start: Run: docker version --format {{.Server.APIVersion}}
[5442 ms] Server API version: 1.43
[5453 ms] Start: Run: tar --no-same-owner -x -f -
[5765 ms] Start: Run: docker build -f C:\Users\<MYUSER>\AppData\Local\Temp\vsch\bootstrap-image\0.298.0\bootstrap.Dockerfile -t vsc-volume-bootstrap C:\Users\<MYUSER>\AppData\Local\Temp\vsch\bootstrap-image\0.298.0
[+] Building 20.0s (7/9)
=> [internal] load build definition from bootstrap.Dockerfile 0.1s
=> => transferring dockerfile: 570B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for mcr.microsoft.com/devcontainers/base:0-alpine-3.16 1.3s
=> [1/5] FROM mcr.microsoft.com/devcontainers/base:0-alpine-3.16@sha256:5afcbaa7ff3ec87bdf5433c8e5aee85aa278897b77df3e7cc22bfa0a11888626 16.5s
=> => resolve mcr.microsoft.com/devcontainers/base:0-alpine-3.16@sha256:5afcbaa7ff3ec87bdf5433c8e5aee85aa278897b77df3e7cc22bfa0a11888626 0.1s
=> => sha256:5afcbaa7ff3ec87bdf5433c8e5aee85aa278897b77df3e7cc22bfa0a11888626 1.61kB / 1.61kB 0.0s
=> => sha256:043a8cc100a674ee748a9e193345226db80dd21d9bed71291e12a0b7c973d6dc 9.91kB / 9.91kB 0.0s
=> => sha256:5c96813e90f548aea9ac9775a432fdbc9ebb3468f1aae3afa61c58dcbdc6c4e5 134B / 134B 0.4s
=> => sha256:7b332c289ca3e30771021443061131372e888102cbd5b5079cca1c853583829e 1.81kB / 1.81kB 0.0s
=> => sha256:99d2f8ff458abdfe87b6a3d430844cdf20d10d25159aa7cf5b1d78928f9a4c8e 408B / 408B 0.3s
=> => sha256:b9915a9e4f02b5b43b882c62e9962aa20c53c7b5213efde856984ac77b455a1f 223B / 223B 0.5s
=> => extracting sha256:99d2f8ff458abdfe87b6a3d430844cdf20d10d25159aa7cf5b1d78928f9a4c8e 0.0s
=> => sha256:56af547a5ec059de387f2c398b8e420859aa97858d16058af66c57b66e40f0c4 232B / 232B 0.6s
=> => extracting sha256:5c96813e90f548aea9ac9775a432fdbc9ebb3468f1aae3afa61c58dcbdc6c4e5 0.0s
=> => extracting sha256:b9915a9e4f02b5b43b882c62e9962aa20c53c7b5213efde856984ac77b455a1f 0.0s
=> => sha256:72b927c65044ab696ee2e2eb22f24a76be9acdb3e6804c93f07c0ead34a22253 189.69MB / 189.69MB 8.5s
=> => extracting sha256:56af547a5ec059de387f2c398b8e420859aa97858d16058af66c57b66e40f0c4 0.0s
=> => sha256:5ec70e3ccad0b9d2f6f1f851bffb53e13f358b845b18dc9bb5bebbff4478d53d 38.68MB / 38.68MB 3.1s
=> => extracting sha256:72b927c65044ab696ee2e2eb22f24a76be9acdb3e6804c93f07c0ead34a22253 5.3s
=> => extracting sha256:5ec70e3ccad0b9d2f6f1f851bffb53e13f358b845b18dc9bb5bebbff4478d53d 2.4s
=> [internal] load build context 0.3s
=> => transferring context: 1.35MB 0.2s
=> [2/5] RUN echo "@old https://dl-cdn.alpinelinux.org/alpine/v3.15/main" >> /etc/apk/repositories 1.3s
=> ERROR [3/5] RUN apk add --no-cache git-lfs nodejs python3 npm make g++ docker-cli docker-cli-buildx docker-cli-compose openssh-client-default@old ; 0.8s
------
> [3/5] RUN apk add --no-cache git-lfs nodejs python3 npm m
ake g++ docker-cli docker-cli-buildx docker-cli-compose openssh-client-default@old ;
:
#0 0.531 fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
#0 0.656 139808755891016:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1919:
#0 0.656 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.16/main: Permission denied
#0 0.656 fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
#0 0.719 139808755891016:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1919:
#0 0.720 fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
#0 0.720 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.16/community: Permission denied
#0 0.782 139808755891016:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1919:
#0 0.783 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.15/main: Permission denied
#0 0.784 WARNING: The repository tag for world dependency 'openssh-client-default@old' does not exist
#0 0.784 ERROR: Not committing changes due to missing repository tags. Use --force-broken-world to override.
------
bootstrap.Dockerfile:6
--------------------
5 |
6 | >>> RUN apk add --no-cache \
7 | >>> git-lfs \
8 | >>> nodejs \
9 | >>> python3 \
10 | >>> npm \
11 | >>> make \
12 | >>> g++ \
13 | >>> docker-cli \
14 | >>> docker-cli-buildx \
15 | >>> docker-cli-compose \
16 | >>> openssh-client-default@old \
17 | >>> ;
18 |
--------------------
ERROR: failed to solve: process "/bin/sh -c apk add --no-cache \tgit-lfs \tnodejs \tpython3 \tnpm \tmake \tg++ \tdocker-cli \tdocker-cli-buildx \tdocker-cli-compose \topenssh-
client-default@old \t;" did not complete successfully: exit code: 99
[29876 ms] Command failed: docker build -f C:\Users\<MYUSER>\AppData\Local\Temp\vsch\bootstrap-image\0.298.0\bootstrap.Dockerfile -t vsc-volume-bootstrap C:\Users\<MYUSER>\AppData\Local\Temp\vsch\bootstrap-image\0.298.0
[29876 ms] Exit code 1