Host Networking feature does not properly identity all ports to bind to Mac host

[dcarbone]

Description

Host Information:

System Version:     macOS 12.7.5(21H1222)
Machine Identifier: MacBookPro13,2

I have a go binary that runs inside an Alpine container. When developing locally, this container must be run with host networking. On my primary Ubuntu development machine, I am able to build the listener using 0.0.0.0:$PORT or 127.0.0.1:$PORT. On this Mac that I was provided to test updates I make for Mac user compatibility, the only way I am able to get Docker Desktop to bind the host's loopback interface with the expected port(s) is to specifically bind them to 127.0.0.1:$PORT. Using 0.0.0.0:$PORT does not work.

Attached is code I used to reproduce this issue in isolation: mac_dd_nethost_bind_test.zip

The original source code demonstrates what happens with 0.0.0.0:9002 as the listener bind address. This does not propagate to the host loopback interface on macos. To see a working example, you will need to edit the provided main.go file line 28 from 0.0.0.0:9002 to 127.0.0.1:9002, rebuild and re-run.

Reproduce

1. Download & unzip attached code
2. cd into expanded archive dir

To see outcome of 0.0.0.0 bind:

1. docker build -t mac_dd_nethost_test --load .
2. docker run --rm --net=host mac_dd_nethost_test
3. Attempt to open browser to http://127.0.0.1:9002

To see outcome of 127.0.0.1 bind:

1. Edit line 28 of attached main.go, replacing 0.0.0.0 with 127.0.0.1
2. Repeat steps 1-3 above.

Expected behavior

Creating a listener that binds 0.0.0.0 should result in the in-container binds being propagated to the macos host loopback interface.

docker version

Client:
 Version:           26.1.4
 API version:       1.45
 Go version:        go1.21.11
 Git commit:        5650f9b
 Built:             Wed Jun  5 11:26:02 2024
 OS/Arch:           darwin/amd64
 Context:           desktop-linux

Server: Docker Desktop 4.31.0 (153195)
 Engine:
  Version:          26.1.4
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.11
  Git commit:       de5c9cf
  Built:            Wed Jun  5 11:29:22 2024
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.33
  GitCommit:        d2d58213f83a351ca8f528a95fbd145f5654e957
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    26.1.4
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.1-desktop.1
    Path:     $HOME/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.1-desktop.1
    Path:     $HOME/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.32
    Path:     $HOME/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     $HOME/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.24
    Path:     $HOME/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     $HOME/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.2.0
    Path:     $HOME/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     $HOME/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.9.3
    Path:     $HOME/.docker/cli-plugins/docker-scout

Server:
 Containers: 3
  Running: 1
  Paused: 0
  Stopped: 2
 Images: 18
 Server Version: 26.1.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d2d58213f83a351ca8f528a95fbd145f5654e957
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.31-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.823GiB
 Name: docker-desktop
 ID: a85fcfab-bc95-4b91-8463-9118c98f17d4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix://$HOME/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

3935486C-8631-4461-B3FB-19E3AE42907D/20240708172250

Additional Info

No response

[dcarbone]

lsof output with sample code:

When binding 0.0.0.0:

# sudo lsof -nP -i4TCP | grep LISTEN | grep 9002
#

(results in empty output, exit code 1)

When binding 127.0.0.1:

# sudo lsof -nP -i4TCP | grep LISTEN | grep 9002
com.docke 5874 dcarbone  177u  IPv4 0x7f1382407af2058d      0t0  TCP 127.0.0.1:9002 (LISTEN)
#

(results in 1 row returned, exit code 0)

[akerouanton]

I can repro this issue on the latest nightly build of Docker Desktop. I also tested it with my wip branch where I add support for v6-only and dual-stack, and I can confirm it doesn't suffer from this bug.

I'm pretty sure this bug has already been reported but I can't find the original ticket. I'll close this one as duplicate if I can get hold of it.

For DD versions released up until now, the workaround is to make sure the socket is bound to the IPv4 ANY address.

diff --git a/main.go b/main.go
index f00ac31..a21f667 100644
--- a/main.go
+++ b/main.go
@@ -25,7 +25,7 @@ func main() {
 		_, _ = w.Write([]byte("hellord"))
 	})
 
-	if l, err = lcfg.Listen(ctx, "tcp", "0.0.0.0:9002"); err != nil {
+	if l, err = lcfg.Listen(ctx, "tcp4", "0.0.0.0:9002"); err != nil {
 		panic(err.Error())
 	}
 

The provided repro makes a call to ListenConfig.Listen(), which get passed tcp as its 2nd argument. net.Dial() defines the semantic of that argument:

Known networks are "tcp", "tcp4" (IPv4-only), "tcp6" (IPv6-only)

That means tcp is dual-stack. This can be observed with strace:

# With tcp
$ strace -f --trace=bind ./testapp
...
[pid  1615] bind(3, {sa_family=AF_INET6, sin6_port=htons(9002), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::", &sin6_addr), sin6_scope_id=0}, 28) = 0

# With tcp4
$ strace -f --trace=bind ./testapp
[pid  1392] bind(3, {sa_family=AF_INET, sin_port=htons(9002), sin_addr=inet_addr("0.0.0.0")}, 16) = 0

[akerouanton]

@dcarbone IPv6 / dual-stack support was added to the host networkwing feature in the latest Docker Desktop release (4.33.0). The workaround above should not be needed anymore.

I'll close this issue as completed, but feel free to continue the discussion and ping me if you still face some issues.