Port-mapping containers do not start after ungraceful shutdown

[ghost]

This ticket resembles tickets from 2016 and earlier. For various reasons those are closed. My apologies if this is a duplicate or the problem is fixed in a newer version or the bug belongs to a different repository. I must have missed something while browsing the project page.

Abruptly power-cycle a host, and port-mapping containers fail to start. This does not always happen. The docker-proxy will be listening on the relevant port. A workaround is to gracefully restart the host. Depending upon the root cause, the second restart may work sometimes.

The error message is as follows. "starting container failed: container 3180fc4b8d11cc4970a41c05af3f4062018b7c8c4b5946cf88c56f1abff5c1c6: endpoint join on GW Network failed: driver failed programming external connectivity on endpoint gateway_3180fc4b8d11 (e2843d2a8d7b394f1f0c0cc5213bc4af9cdc4ebb6f837849b3ea069ff82f16a1): Bind for 0.0.0.0:18246 failed: port is already allocated"

The output of netstat is as follows.

/ # netstat -planu
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
...
udp        0      0 :::18246                :::*                                674/docker-proxy

The iptables output on the host contains the following, even though the docker container which listens on the port fails to come up.

root@predixedge:~# iptables -L -t nat --line-numbers
Chain DOCKER (2 references)
num  target     prot opt source               destination         
1    DNAT       tcp  --  anywhere             172.17.0.1           tcp dpt:53 to:127.0.0.2:53
2    DNAT       udp  --  anywhere             172.17.0.1           udp dpt:53 to:127.0.0.2:53
3    RETURN     all  --  anywhere             anywhere            
4    RETURN     all  --  anywhere             anywhere            
5    DNAT       udp  --  anywhere             anywhere             udp dpt:18246 to:172.18.0.2:18246

The output of "docker version" is:

root@predixedge:~# docker version
Client:
 Version:	18.03.0
 API version:	1.37
 Go version:	go1.9.7
 Git commit:	0f1bb35
 Built:	Sat Aug 24 19:38:46 2019
 OS/Arch:	linux/amd64
 Experimental:	false
 Orchestrator:	swarm

Server:
 Engine:
  Version:	18.03.0
  API version:	1.37 (minimum version 1.12)
  Go version:	go1.9.7
  Git commit:	708b068d3095c6a6be939eb2da78c921d2e945e2
  Built:	Sat Aug 24 19:35:44 2019
  OS/Arch:	linux/amd64
  Experimental:	false