for-mac icon indicating copy to clipboard operation
for-mac copied to clipboard

4.30.0 update breaks VScode devcontainer auto port forwarding

Open darthtechworker opened this issue 1 year ago • 17 comments

Description

4.30.0 breaks auto port forwarding in VScode dev containers. With 4.29.0 (or whatever was before 4.30.0/today's update) I could start my dev container and run npm run dev and access my website via http://localhost:5173 in the browser. That does not work anymore.

I can make it work by doing npm run dev -- --host (but I did not have to do that before today's update and VScode did not release any updates today for this).

Reproduce

  1. Have Docker Desktop 4.30.0 running
  2. Clone any basic react + vite repo (sample)
  3. Open it in VScode
  4. Open Command Palette (Cmd + Shift + P) and select Dev Containers: Rebuild Container
  5. Once the container is done building:
  6. run npm i inside the container
  7. run rpm run dev inside the container
  8. Go to http://localhost:5173 in your browser

Expected behavior

A simple react website should load.

docker version

Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:04 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    26.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0-desktop.1
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.0-desktop.2
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.29
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.23
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.1.0
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.8.0
    Path:     /Users/gauravpandey/.docker/cli-plugins/docker-scout

Server:
 Containers: 3
  Running: 1
  Paused: 0
  Stopped: 2
 Images: 13
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.26-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 11
 Total Memory: 7.657GiB
 Name: docker-desktop
 ID: 4f59a7ce-876f-467d-8d5c-be13a4c06274
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/gauravpandey/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

BC1CB2B2-B69E-4396-8D7A-B7A17288F241/20240510045331

Additional Info

No response

darthtechworker avatar May 10 '24 04:05 darthtechworker

I tried the same on another machine with docker 4.29.0 and this works as expected. For further testing, I also upgraded the docker desktop in this machine to 4.30.0 and I see the issue as described above. So, clearly this bug was introduced in 4.30.0. Was any changes made on how host network interacts with docker network?

darthtechworker avatar May 12 '24 18:05 darthtechworker

cc @djs55

dgageot avatar May 13 '24 09:05 dgageot

This happens when pairing docker desktop 4.30.0 with vscode devcontainers on Windows too.

biggyspender avatar May 13 '24 18:05 biggyspender

Docker Desktop on window updated to v4.30.0 this morning, and can confirm that port forwarding is no longer working

Update: a downgrade to 4.29.0 has vscode devcontainer port forwarding working again. After rebuilding the containers.

adamdva avatar May 14 '24 15:05 adamdva

Downgrading to 4.29.0 or 4.27.2 quickly causes docker desktop to automatically exit, uninstall and install the latest 4.30.0 even with "Automatically check for updates" and "Always download updates" unchecked.

casche avatar May 15 '24 12:05 casche

We are facing the same with both docker desktop on Windows and Linux. Downgrading to 4.29.0 fixed it.

ijaouani avatar May 22 '24 15:05 ijaouani

Facing the same issues on Docker for Windows latest version 4.31.0

kleozzy avatar Jun 10 '24 07:06 kleozzy

I can confirm that 4.31.0 on Mac has this issue. Rolling back to 4.29.0 is a workaround.

profbbrown avatar Jun 12 '24 01:06 profbbrown

Another workaround is adding the following to the vite config:

server: {
  host: '127.0.0.1'
}

vshah505 avatar Jun 13 '24 20:06 vshah505

Same here, on MacOs, can confirm 4.31.0 same issue, downgrade to 4.29.0 did the trick as a workaround

mehd-io avatar Jun 15 '24 12:06 mehd-io

I tried it with 4.32.0 on MacOS and get the same issue, the port forwarding does not work.

mezeitamas avatar Jul 10 '24 14:07 mezeitamas

Not related to Mac, but I faced the same issue on Docker for Windows 4.32.0.

ibara1454 avatar Jul 10 '24 18:07 ibara1454

Same issue here, Mac, version 4.32.0 and 4.31.1

bsuligoj avatar Jul 16 '24 07:07 bsuligoj

Another workaround is adding the following to the vite config:

server: {
  host: '127.0.0.1'
}

Or angular: ng serve --host 127.0.0.1

Version 4.32.0 (157355).

Remark: the open port is not displayed in Docker Desktop (Mac) in the container list, but it works...

hlogmans avatar Aug 19 '24 14:08 hlogmans

The issue occurs in Windows 22631.4037 with Docker version 4.33.1. I need to rollback to Docker ver. 4.29.0

cloud-practitioner avatar Aug 22 '24 06:08 cloud-practitioner

Another workaround is adding the following to the vite config:

server: {
  host: '127.0.0.1'
}

Or angular: ng serve --host 127.0.0.1

Version 4.32.0 (157355).

Remark: the open port is not displayed in Docker Desktop (Mac) in the container list, but it works...

Worked for me in Manjaro 24.0.7 and Docker version 27.1.1, build 63125853e3 Also when I do docker ps it's not displaying the forwarded ports.

diegogtus avatar Aug 29 '24 23:08 diegogtus

It's been about six months and this problem remains unfixed without a workaround. Clearly, Docker doesn't think it's their problem. Is it actually a VS Code issue? Or something that needs updating in dev container configurations?

Where do we go from here? If someone hasn't done so, open an issue with VS Code and see if we can get some more traction.

profbbrown avatar Sep 18 '24 03:09 profbbrown

Can confirm that accessing Angular app in devcontainer works only when --host 127.0.0.1 is specified. Docker 4.34 Before update it worked without host specification.

Also I don't think this issue has correct name. Auto port forwarding works ok, it's just that now apps binds to different host I guess.

jakubnavratil avatar Nov 05 '24 14:11 jakubnavratil

Can confirm that switching from "ng serve --port=4000" to "ng serve --port=4000 --host=127.0.0.1" solved the connection issue for Angular using devcontainers in WSL2.

KyrumX avatar Nov 14 '24 16:11 KyrumX

I'm not using Docker Desktop but docker-ce directly in WSL. This doesn't seem to be an issue with Docker Desktop, but with the Docker Engine itself. I tested different versions to pin down when this issue was introduced.

The issue first appeared in 26.0.2, and is present in all versions up to the current version (27.0.5). The last unaffected release is 26.0.1. Note that 26.0.0 and 26.0.1 are affected by CVE-2024-32473, so you might want to downgrade to 25.0.5 instead if this vulnerability is relevant to you.
The fix for this vulnerability might be related to this issue, since it seems that it is caused by VS Code port forwarding being broken with IPv6. Disabling IPv6 in the container is another workaround.

Specifically, I downgraded to the following package on my Ubuntu 22.04 WSL installation:

sudo apt install docker-ce=5:26.0.1-1~ubuntu.22.04~jammy
sudo apt-mark hold docker-ce

The issue has already been reported to vscode-remote: https://github.com/microsoft/vscode-remote-release/issues/7029
I don't know if there is also an issue with https://github.com/moby/moby, since it was working in Docker 25 and earlier, and some change in 26.0.2 broke something.

EDIT:
After further research:

  • It worked in Version 25 and earlier because IPv6 was disabled in containers in more cases (https://github.com/vitejs/vite/issues/16522#issuecomment-2084322973). Applications binding to localhost in containers were listening on 127.0.0.1.
  • 26.0.0 changed that and enabled support for IPv6 in containers even if the network did not have IPv6 enabled. Due to a bug, IPv6 was still enabled for those networks anyways. Applications binding to localhost are now binding to [::1], but due to the bug connecting from outside the container to localhost via IPv6 still worked, even though the port was never forwarded. This bug caused CVE-2024-32473.
  • 26.0.2 fixed this bug. Applications are still binding to [::1], but the network interface to the host now correctly has IPv6 disabled, so the application is not reachable from the outside. Even if it was enabled, VS Code does not support auto port forwarding for IPv6 (https://github.com/microsoft/vscode-remote-release/issues/7029).

Workarounds:

  • Explicitly tell the application to bind to the IPv4 address with --host 127.0.0.1 or similar options (depending on the application). Some applications have already changed the default binding from localhost to 127.0.0.1 work around this issue.
  • Disable IPv6 in the container with the run arg --sysctl net.ipv6.conf.all.disable_ipv6=1 or the following docker compose config:
      sysctls:
        - net.ipv6.conf.all.disable_ipv6=1
    
    This removes the IPv6 entries in /etc/hosts inside the container, so applications binding to localhost will bind to 127.0.0.1 again.

ReneZeidler avatar Jan 23 '25 14:01 ReneZeidler

Starting with v26.0 Docker Engine always inject ::1 into /etc/hosts, so applications that resolve localhost will get this IP address returned and might listen on IPv6 instead of IPv4. Most of the time, it should not cause any issue, except if some components are making invalid assumptions about localhost always being 127.0.0.1.

Looking at VSCode logs in ~/Library/Application\ Support/Code/logs/ I can see the following errors. This clearly shows that for some reason VSCode totally ignores the IP address used by the listening socket, and decide that it should forward to 127.0.0.1 instead of ::1:

[2025-01-27T09:12:52.167Z] rejected promise not handled within 1 second: Error: connect ECONNREFUSED 127.0.0.1:5175
stack trace: Error: connect ECONNREFUSED 127.0.0.1:5175
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16)
[2025-01-27T09:12:52.168Z] [09:12:52] Error: connect ECONNREFUSED 127.0.0.1:5175
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) {
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 5175
}
rejected promise not handled within 1 second: Error: connect ECONNREFUSED 127.0.0.1:5175
stack trace: Error: connect ECONNREFUSED 127.0.0.1:5175
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16)
[09:12:52] Error: connect ECONNREFUSED 127.0.0.1:5175
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) {
  errno: -111,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 5175
}

To be clear, we're not going to change the Engine behavior because: 1. it's perfectly fine, at least if no software make wrong assumptions; 2. reverting back to what we had prior to v26 would break people that now make the assumption that localhost is IPv6.

For now, you need to either disable IPv6 within your container with --sysctl net.ipv6.conf.all.disable_ipv6=1, or configure vite to listen on either 127.0.0.1, 0.0.0.0 or ::.

I'd recommend that whoever is facing this issue should report it to the VSCode team.

akerouanton avatar Jan 27 '25 09:01 akerouanton

One of the vite maintainer went ahead and added a section to vite's troubleshooting guide stating what the issue is and how to work around it. See: https://github.com/vitejs/vite/pull/19303

Since the issue isn't on our side, I'll close this ticket but feel free to continue the conversation.

akerouanton avatar Jan 27 '25 14:01 akerouanton