cli icon indicating copy to clipboard operation
cli copied to clipboard
verified publisher icon docker

After disconnecting and reconnecting a container, the exposed port does not rebind to the new host port

Open xucito opened this issue 5 years ago • 3 comments

Description After disconnecting and reconnecting a container to the bridge network with a dynamic host port binding, the newly assigned host port is not reachable and the container is still reachable via the old port binding.

Steps to reproduce the issue:

  1. Run docker run --name mynginx1 -p 80 -d nginx
  2. Inspect the container and telnet the binded port
  3. Run docker network disconnect bridge <container Id>
  4. Run docker network connect bridge <container Id>

Describe the results you received:

a. Try to telnet container on newly binded host port (Identify it is unreachable) b. Try to telnet container on old binded host port (Identify it is reachable)

Describe the results you expected: a. Try to telnet container on newly binded host port (Identify it is reachable) b. Try to telnet container on old binded host port (Identify it is unreachable)

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:00:27 2020
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 10
  Running: 4
  Paused: 0
  Stopped: 6
 Images: 1755
 Server Version: 19.03.13
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.104-microsoft-standard
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 24.95GiB
 Name: docker-desktop
 ID: TAE7:3G7R:YSMF:AJDH:OPMK:CFTK:74OY:LJ3M:OFOH:KI4S:TL4I:CXO7
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 74
  Goroutines: 71
  System Time: 2021-01-04T20:38:06.7034025Z
  EventsListeners: 2
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Additional environment details (AWS, VirtualBox, physical, etc.): We are having the same behavior across different types of host OS

xucito avatar Jan 04 '21 20:01 xucito

going through tickets, and I can reproduce this;

  • Looks to be only with randomly (ephemeral) ports
  • :information_source: Situation is slightly different on Docker Desktop, likely because of extra proxies in between (host <--> VM)
  • :question: not sure what the expected behavior is on this one; it may be because the ephemeral port is re-selected on connect?
  • :warning: but on Desktop, there may be a bug because the port doesn't seem to be released (while disconnected)

1. Docker Desktop 4.41.0 (190386), manually defined port-mapping;

docker run --name mynginx1 -p 8080:80 -d nginx:alpine
9e072c150b63983f90a3e352fa1bf517dcd4db1dcb9a67063f7a5ec325b0a511

curl -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 21:42:00 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

docker network disconnect bridge mynginx1

curl -I localhost:8080
curl: (56) Recv failure: Connection reset by peer

docker network connect bridge mynginx1

curl -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 21:42:48 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

docker rm -fv mynginx1

curl -I localhost:8080
curl: (7) Failed to connect to localhost port 8080 after 0 ms: Couldn't connect to server

2. Docker Desktop 4.41.0 (190386), randomly assigned (ephemeral) port;

docker run --name mynginx1 -p 80 -d nginx:alpine

docker port mynginx1 80
0.0.0.0:49511

curl -I localhost:49511
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 21:45:23 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

disconnecting; note that the port appears to still be mapped, but doesn't give a response (unlike for the manually mapped port, which gave curl: (56) Recv failure: Connection reset by peer); `

docker network disconnect bridge mynginx1

curl -I localhost:49511
curl: (52) Empty reply from server

re-connecting to the network;

docker network connect bridge mynginx1

curl -I localhost:49511
curl: (52) Empty reply from server

removing the container removes the port;

docker rm -fv mynginx1

curl -I localhost:49511
curl: (7) Failed to connect to localhost port 49511 after 0 ms: Couldn't connect to server

3. On Ubuntu 24.10, manually defined port-mapping

docker run --name mynginx1 -p 8080:80 -d nginx:alpine

curl -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 21:59:55 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

disconnect:

docker network disconnect bridge mynginx1
curl -I localhost:8080
curl: (7) Failed to connect to localhost port 8080 after 0 ms: Couldn't connect to server

re-connect:

docker network connect bridge mynginx1
curl -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 22:00:50 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

remove:

docker rm -fv mynginx1

curl -I localhost:8080
curl: (7) Failed to connect to localhost port 8080 after 0 ms: Couldn't connect to server

4. On Ubuntu 24.10, random (ephemeral) port

docker run --name mynginx1 -p 80 -d nginx:alpine

docker port mynginx1 80
0.0.0.0:32768
[::]:32768

curl -I localhost:32768
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 21:52:55 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

disconnecting; note that the port is not mapped (failed to connect);

curl -I localhost:32768
curl: (7) Failed to connect to localhost port 32768 after 0 ms: Couldn't connect to server

reconnecting;

docker network connect bridge mynginx1
curl -I localhost:32768
curl: (7) Failed to connect to localhost port 32768 after 0 ms: Couldn't connect to server

removing;

docker rm -fv mynginx1

curl -I localhost:32768
curl: (7) Failed to connect to localhost port 32768 after 0 ms: Couldn't connect to server

thaJeztah avatar Apr 24 '25 22:04 thaJeztah

❓ not sure what the expected behavior is on this one; it may be because the ephemeral port is re-selected on connect?

Confirming that that's indeed the cause, so I think this may be expected behavior; ephemeral ports are assigned when connecting to the network. When disconnecting, that port is released, and when re-attachiing to the network, a new port is selected;

First time, the port is 32770;

docker run --name mynginx1 -p 80 -d nginx:alpine
docker port mynginx1 80
0.0.0.0:32770
[::]:32770

curl -I localhost:32770
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 22:07:34 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

After disconnecting, no port is assigned;

docker network disconnect bridge mynginx1
docker port mynginx1 80
Error: No public port '80' published for mynginx1

Re-attaching to the network assigns a new port (32771);

docker network connect bridge mynginx1
docker port mynginx1 80
0.0.0.0:32771
[::]:32771

And connecting to the new port works;

curl -I localhost:32771
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 22:08:19 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

Docker Desktop (bug?)

On Docker Desktop, it looks like there's a bug; port-mapping isn't restored / gets out of sync;

First steps look correct;

docker run --name mynginx1 -p 80 -d nginx:alpine

docker port mynginx1 80
0.0.0.0:50356

curl -I localhost:50356
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 22:25:39 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

Inspecting the container also shows the port mapped;

docker inspect --format '{{json .NetworkSettings.Ports}}' mynginx1
{"80/tcp":[{"HostIp":"0.0.0.0","HostPort":"50356"}]}

And so does docker ps

docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS          PORTS                   NAMES
2b2eda5e33a8   nginx:alpine   "/docker-entrypoint.…"   51 seconds ago   Up 50 seconds   0.0.0.0:50356->80/tcp   mynginx1

However, after disconnecting, Docker Desktop's port-mapping gets confused; the container no longer has a port mapped, but Docker Desktop is still listening on the port;

docker network disconnect bridge mynginx1
docker port mynginx1 80
Error: No public port '80' published for mynginx1

curl -I localhost:50356
curl: (52) Empty reply from server

Inspecting the container looks correct;

docker inspect --format '{{json .NetworkSettings.Ports}}' mynginx1
{}

When re-connecting to the network, things get more confusing; docker port doesn't show a port mapped, but also doesn't show an error ("No public port '80' published for mynginx1");

docker network connect bridge mynginx1
docker port mynginx1 80
# no response here

docker port mynginx1
# no response here

Inspecting the container show the port, but not mapped;

docker inspect --format '{{json .NetworkSettings.Ports}}' mynginx1
{"80/tcp":[]}

But docker ps DOES show a port;

docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED         STATUS         PORTS                     NAMES
2b2eda5e33a8   nginx:alpine   "/docker-entrypoint.…"   5 minutes ago   Up 5 minutes   127.0.0.1:55008->80/tcp   mynginx1

But it's not mapped;

curl -I localhost:55008
curl: (7) Failed to connect to localhost port 55008 after 0 ms: Couldn't connect to server

Trying a stop, start of the container looks to restore the expected behavior; a new port is assigned, and shows up in both docker ps, docker port, and docker inspect;

docker stop mynginx1
docker start mynginx1

docker port mynginx1 80
0.0.0.0:50431

docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED         STATUS          PORTS                   NAMES
2b2eda5e33a8   nginx:alpine   "/docker-entrypoint.…"   6 minutes ago   Up 27 seconds   0.0.0.0:50431->80/tcp   mynginx1

docker inspect --format '{{json .NetworkSettings.Ports}}' mynginx1
{"80/tcp":[{"HostIp":"0.0.0.0","HostPort":"50431"}]}

And after this, I can connect again;

curl -I localhost:50431
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Thu, 24 Apr 2025 22:32:43 GMT
Content-Type: text/html
Content-Length: 615
Last-Modified: Wed, 16 Apr 2025 12:55:34 GMT
Connection: keep-alive
ETag: "67ffa8c6-267"
Accept-Ranges: bytes

thaJeztah avatar Apr 24 '25 22:04 thaJeztah

@akerouanton @robmry PTAL 😄

thaJeztah avatar Apr 24 '25 22:04 thaJeztah