Watchtower fails to update itself when API is enabled

[piksel]

Describe the bug

Watchtower is unable to recreate itself when it's forwarding ports to the OS. The new container is unable to be created due to the requested port being in use.

Steps to reproduce

1. Make sure the local containrrr/watchtower:latest points to a older version of watchtower, this can be accomplished by:

   docker pull containrrr/watchtower:1.5.0
   docker image tag containrrr/watchtower:1.5.0 containrrr/watchtower:latest
   

2. Run watchtower with port forwarding:

   docker run --rm -it -p 8080:8080 \
     -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --run-once --debug
   

3. Watchtower fails to update itself.

Expected behavior

Watchtower is able to update itself... somehow.

Screenshots

No response

Environment

• Docker Desktop / WSL2
• AMD64
• Docker v20.10.21

Your logs

DEBU[0000] Sleeping for a second to ensure the docker api client has been properly initialized.
DEBU[0001] Making sure everything is sane before starting
INFO[0001] Watchtower 1.5.0
INFO[0001] Using no notifications
INFO[0001] Checking all containers (except explicitly disabled with label)
INFO[0001] Running a one time update.
DEBU[0001] Checking containers for updated images
DEBU[0001] Retrieving running containers
DEBU[0001] Trying to load authentication credentials.    container=/silly_allen image="containrrr/watchtower:latest"
DEBU[0001] No credentials for containrrr found           config_file=/config.json
DEBU[0001] Got image name: containrrr/watchtower:latest
DEBU[0001] Checking if pull is needed                    container=/silly_allen image="containrrr/watchtower:latest"
DEBU[0001] Building challenge URL                        URL="https://index.docker.io/v2/"
DEBU[0001] Got response to challenge request             header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\"" status="401 Unauthorized"
DEBU[0001] Checking challenge header content             realm="https://auth.docker.io/token" service=registry.docker.io
DEBU[0001] Setting scope for auth token                  image=containrrr/watchtower scope="repository:containrrr/watchtower:pull"
DEBU[0001] No credentials found.
DEBU[0001] Parsing image ref                             host=index.docker.io image=containrrr/watchtower normalized="docker.io/containrrr/watchtower:latest" tag=latest
DEBU[0001] Doing a HEAD request to fetch a digest        url="https://index.docker.io/v2/containrrr/watchtower/manifests/latest"
DEBU[0002] Found a remote digest to compare with         remote="sha256:897304ffb41533954deda3ca9dd140fa1ca41e5d7e0bc6d6352606931145779c"
DEBU[0002] Comparing                                     local="sha256:3c6066d0e6bff5c6c64ca4d9d5b4e99fe1a9f462dab36001efb2bd2b652b4750" remote="sha256:897304ffb41533954deda3ca9dd140fa1ca41e5d7e0bc6d6352606931145779c"
DEBU[0002] Digests did not match, doing a pull.
DEBU[0002] Pulling image                                 container=/silly_allen image="containrrr/watchtower:latest"
INFO[0003] Found new containrrr/watchtower:latest image (d6b5c7ec581c)
DEBU[0003] This is the watchtower container /silly_allen
DEBU[0003] Renaming container /silly_allen (81912a51dc0d) to XVlBzgbaiCMRAjWwhTHctcuAxhxKQFDa
INFO[0003] Creating /silly_allen
DEBU[0003] Starting container /silly_allen (2fa9af5c5f6a)
ERRO[0003] Error response from daemon: driver failed programming external connectivity on endpoint silly_allen (2f06d54fc82de28d1734d8fa8b03ca53c5b7ee265634dd5c651eb8459facdcae): Bind for 0.0.0.0:8080 failed: port is already allocated
INFO[0003] Session done                                  Failed=1 Scanned=1 Updated=0 notify=no
INFO[0003] Waiting for the notification goroutine to finish  notify=no

Additional context

Started using

docker run --rm -it -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --http-api-metrics --http-api-token t --interval 10

[barcar]

Is there a plan to fix this or is there a workaround? Thanks.

[piksel]

I have been thinking about how to solve this, but have not come up with a non-janky solution yet...

Either:

1. Start new watchtower with no port forwarding and some environment variable containing the port forwarding config. Then in the new container do another update of itself, resetting the port forwarding config.
2. Spawn another watchtower instance when there is a new update (at the end of the current session) with --run-once and the current container instance name as its arguments. This would only recreate the watchtower instance and then exit.

Both seem like they can create additional problems though...

[barcar]

Could the API pause a few minutes before it starts listening. Would that allow the updated Watchtower to come up and the old one to shut down before the API kicks in and opens the port?

[piksel]

No, because it's the docker port forwarding that is the problem. Two containers cannot have ports forwarded to the same port on the host.