[BUG] docker compose up --watch with sync+restart ends up disconnecting from watch
Description
Similar to https://github.com/docker/compose/issues/11773 I'm seeing docker compose up --watch disconnecting when the container is restarted:
❯ docker compose up --watch
[+] Running 1/0
✔ Container caddy-caddy-1 Running 0.0s
⦿ Watch enabled
Attaching to caddy-1
⦿ Syncing and restarting service "caddy" after 1 changes were detected
caddy-1 | 2024/11/23 00:42:30.609 INFO shutting down apps, then terminating {"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:30.609 WARN exiting; byeee!! 👋 {"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:30.609 INFO http servers shutting down with eternal grace period
caddy-1 | 2024/11/23 00:42:30.609 INFO admin stopped previous server {"address": "localhost:2019"}
caddy-1 | 2024/11/23 00:42:30.609 INFO shutdown complete {"signal": "SIGTERM", "exit_code": 0}
⦿ service "caddy" restarted
caddy-1 exited with code 0
⦿ Syncing and restarting service "caddy" after 1 changes were detected
caddy-1 | 2024/11/23 00:42:39.434 INFO shutting down apps, then terminating {"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:39.434 WARN exiting; byeee!! 👋 {"signal": "SIGTERM"}
caddy-1 | 2024/11/23 00:42:39.434 INFO http servers shutting down with eternal grace period
caddy-1 | 2024/11/23 00:42:39.434 INFO admin stopped previous server {"address": "localhost:2019"}
caddy-1 | 2024/11/23 00:42:39.434 INFO shutdown complete {"signal": "SIGTERM", "exit_code": 0}
caddy-1 exited with code 0
⦿ Watch disabled
❯
services:
caddy:
image: torarnv/caddy
build: .
restart: unless-stopped
ports:
- "8080:80"
develop:
watch:
- action: sync+restart
path: ./Caddyfile
target: /etc/caddy/Caddyfile
Likely because it temporarily sees that there are no services running.
Adding a keep-alive workaround service "fixes" the issue:
dummy-watch-workaround:
image: alpine:latest
init: true
command: ["sh", "-c", "while true; do sleep 2; done"]
Compose Version
2.30.3
I tried to reproduce without success.
I'm using:
services:
app1:
build:
dockerfile_inline: FROM nginx
develop:
watch:
- action: sync+restart
path: .
target: /tmp
Can you please give it a try?
@ndeloof this still reproduces for me:
services:
caddy-reproducer:
image: caddy:2.9.1
restart: unless-stopped
ports:
- "80:80"
- "443:443"
- "443:443/udp"
develop:
watch:
- action: sync+restart
path: .
target: /tmp
❯ docker compose up --watch 8s
[+] Running 1/1
✔ Container caddyrepo-caddy-reproducer-1 Recreated 0.2s
⦿ Watch enabled
Attaching to caddy-reproducer-1
caddy-reproducer-1 | {"level":"warn","ts":1742049446.3041916,"msg":"failed to set GOMAXPROCS","error":"open /sys/fs/cgroup/cpu/cpu.cfs_quota_us: no such file or directory"}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3048794,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3058498,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3072276,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-reproducer-1 | {"level":"warn","ts":1742049446.3081517,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3084567,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000145480"}
caddy-reproducer-1 | {"level":"warn","ts":1742049446.308958,"logger":"http","msg":"HTTP/2 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1 | {"level":"warn","ts":1742049446.308987,"logger":"http","msg":"HTTP/3 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3090036,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3100202,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3100362,"msg":"serving initial configuration"}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3114846,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
caddy-reproducer-1 | {"level":"info","ts":1742049446.3122306,"logger":"tls","msg":"finished cleaning storage units"}
⦿ Syncing and restarting service "caddy-reproducer" after 1 changes were detected
caddy-reproducer-1 | {"level":"info","ts":1742049452.269159,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
caddy-reproducer-1 | {"level":"warn","ts":1742049452.2692258,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
caddy-reproducer-1 | {"level":"info","ts":1742049452.2692566,"logger":"http","msg":"servers shutting down with eternal grace period"}
caddy-reproducer-1 | {"level":"info","ts":1742049452.269483,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
caddy-reproducer-1 | {"level":"info","ts":1742049452.2694945,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
caddy-reproducer-1 exited with code 0
❯ docker compose version
Docker Compose version 2.32.1
I can't reproduce :'(
✔ Container truc-caddy-reproducer-1 Created 0.1s
⦿ Watch enabled
Attaching to caddy-reproducer-1
caddy-reproducer-1 | {"level":"info","ts":1742208021.9712799,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-reproducer-1 | {"level":"info","ts":1742208021.9721558,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-reproducer-1 | {"level":"info","ts":1742208021.9731705,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-reproducer-1 | {"level":"warn","ts":1742208021.97328,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
caddy-reproducer-1 | {"level":"info","ts":1742208021.9734535,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400049c780"}
caddy-reproducer-1 | {"level":"warn","ts":1742208021.9735224,"logger":"http","msg":"HTTP/2 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1 | {"level":"warn","ts":1742208021.9735262,"logger":"http","msg":"HTTP/3 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1 | {"level":"info","ts":1742208021.9735277,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-reproducer-1 | {"level":"info","ts":1742208021.9772449,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-reproducer-1 | {"level":"info","ts":1742208021.9772654,"msg":"serving initial configuration"}
caddy-reproducer-1 | {"level":"info","ts":1742208021.9792316,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
caddy-reproducer-1 | {"level":"info","ts":1742208021.980091,"logger":"tls","msg":"finished cleaning storage units"}
⦿ Syncing service "caddy-reproducer" after 1 changes were detected
caddy-reproducer-1 | {"level":"info","ts":1742208586.0151498,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
caddy-reproducer-1 | {"level":"warn","ts":1742208586.0156841,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
caddy-reproducer-1 | {"level":"info","ts":1742208586.0158362,"logger":"http","msg":"servers shutting down with eternal grace period"}
caddy-reproducer-1 | {"level":"info","ts":1742208586.017201,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
caddy-reproducer-1 | {"level":"info","ts":1742208586.017228,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
⦿ service(s) ["caddy-reproducer"] restarted
caddy-reproducer-1 exited with code 0
caddy-reproducer-1 | {"level":"info","ts":1742208586.492455,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-reproducer-1 | {"level":"info","ts":1742208586.4934077,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-reproducer-1 | {"level":"info","ts":1742208586.494336,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-reproducer-1 | {"level":"warn","ts":1742208586.4945173,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
Strange :) In my case I'm running docker compose up --watch on my laptop, but with the docker context pointing to my NAS. Could that affect this somehow?
Hello @torarnv I tried to reproduce your issue, in between we released a signifiant number of Compose release and I'm not able to reproduce your issue
caddy-reproducer-1 | {"level":"info","ts":1744638529.4268618,"logger":"tls","msg":"finished cleaning storage units"}
⦿ Syncing service "caddy-reproducer" after 1 changes were detected
caddy-reproducer-1 | {"level":"info","ts":1744638567.5292418,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
caddy-reproducer-1 | {"level":"warn","ts":1744638567.529311,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
caddy-reproducer-1 | {"level":"info","ts":1744638567.5293336,"logger":"http","msg":"servers shutting down with eternal grace period"}
caddy-reproducer-1 | {"level":"info","ts":1744638567.5295353,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
caddy-reproducer-1 | {"level":"info","ts":1744638567.529549,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
⦿ service(s) ["caddy-reproducer"] restarted
caddy-reproducer-1 exited with code 0
caddy-reproducer-1 | {"level":"info","ts":1744638567.8666615,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
Can you let me know if the latest version of Compose fixed the issue also for you?
Not OP, but I also had this issue, just upgraded from v2.28 to v2.38 and it fixed the issue. docker compose up will stay connected after watch restarts the image.
I have version 2.39.2 and I still have the issue