compose icon indicating copy to clipboard operation
compose copied to clipboard

[BUG] docker compose up --watch with sync+restart ends up disconnecting from watch

Open torarnv opened this issue 1 year ago • 8 comments

Description

Similar to https://github.com/docker/compose/issues/11773 I'm seeing docker compose up --watch disconnecting when the container is restarted:

❯ docker compose up --watch
[+] Running 1/0
 ✔ Container caddy-caddy-1  Running                                                        0.0s
        ⦿ Watch enabled
Attaching to caddy-1
         ⦿ Syncing and restarting service "caddy" after 1 changes were detected
caddy-1  | 2024/11/23 00:42:30.609	INFO	shutting down apps, then terminating	{"signal": "SIGTERM"}
caddy-1  | 2024/11/23 00:42:30.609	WARN	exiting; byeee!! 👋	{"signal": "SIGTERM"}
caddy-1  | 2024/11/23 00:42:30.609	INFO	http	servers shutting down with eternal grace period
caddy-1  | 2024/11/23 00:42:30.609	INFO	admin	stopped previous server	{"address": "localhost:2019"}
caddy-1  | 2024/11/23 00:42:30.609	INFO	shutdown complete	{"signal": "SIGTERM", "exit_code": 0}
         ⦿ service "caddy" restarted
caddy-1 exited with code 0
         ⦿ Syncing and restarting service "caddy" after 1 changes were detected
caddy-1  | 2024/11/23 00:42:39.434	INFO	shutting down apps, then terminating	{"signal": "SIGTERM"}
caddy-1  | 2024/11/23 00:42:39.434	WARN	exiting; byeee!! 👋	{"signal": "SIGTERM"}
caddy-1  | 2024/11/23 00:42:39.434	INFO	http	servers shutting down with eternal grace period
caddy-1  | 2024/11/23 00:42:39.434	INFO	admin	stopped previous server	{"address": "localhost:2019"}
caddy-1  | 2024/11/23 00:42:39.434	INFO	shutdown complete	{"signal": "SIGTERM", "exit_code": 0}
caddy-1 exited with code 0
         ⦿ Watch disabled

❯
services:
  caddy:
    image: torarnv/caddy
    build: .
    restart: unless-stopped
    ports:
      - "8080:80"

    develop:
      watch:
        - action: sync+restart
          path: ./Caddyfile
          target: /etc/caddy/Caddyfile

Likely because it temporarily sees that there are no services running.

Adding a keep-alive workaround service "fixes" the issue:

dummy-watch-workaround:
  image: alpine:latest
  init: true
  command: ["sh", "-c", "while true; do sleep 2; done"]

Compose Version

2.30.3

torarnv avatar Nov 24 '24 12:11 torarnv

I tried to reproduce without success.

I'm using:

services:
  app1:
    build:
      dockerfile_inline: FROM nginx
    develop:
      watch:
        - action: sync+restart
          path: .
          target: /tmp

Can you please give it a try?

ndeloof avatar Nov 25 '24 09:11 ndeloof

@ndeloof this still reproduces for me:

services:
  caddy-reproducer:
    image: caddy:2.9.1
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"

    develop:
      watch:
        - action: sync+restart
          path: .
          target: /tmp
❯ docker compose up --watch                                                                                                                                                                                                                                 8s
[+] Running 1/1
 ✔ Container caddyrepo-caddy-reproducer-1  Recreated                                                                                                                                                                                                      0.2s
        ⦿ Watch enabled
Attaching to caddy-reproducer-1
caddy-reproducer-1  | {"level":"warn","ts":1742049446.3041916,"msg":"failed to set GOMAXPROCS","error":"open /sys/fs/cgroup/cpu/cpu.cfs_quota_us: no such file or directory"}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3048794,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3058498,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3072276,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-reproducer-1  | {"level":"warn","ts":1742049446.3081517,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3084567,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc000145480"}
caddy-reproducer-1  | {"level":"warn","ts":1742049446.308958,"logger":"http","msg":"HTTP/2 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1  | {"level":"warn","ts":1742049446.308987,"logger":"http","msg":"HTTP/3 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3090036,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3100202,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3100362,"msg":"serving initial configuration"}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3114846,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
caddy-reproducer-1  | {"level":"info","ts":1742049446.3122306,"logger":"tls","msg":"finished cleaning storage units"}
                    ⦿ Syncing and restarting service "caddy-reproducer" after 1 changes were detected
caddy-reproducer-1  | {"level":"info","ts":1742049452.269159,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
caddy-reproducer-1  | {"level":"warn","ts":1742049452.2692258,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
caddy-reproducer-1  | {"level":"info","ts":1742049452.2692566,"logger":"http","msg":"servers shutting down with eternal grace period"}
caddy-reproducer-1  | {"level":"info","ts":1742049452.269483,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
caddy-reproducer-1  | {"level":"info","ts":1742049452.2694945,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
caddy-reproducer-1 exited with code 0

torarnv avatar Mar 15 '25 14:03 torarnv

❯ docker compose version
Docker Compose version 2.32.1

torarnv avatar Mar 15 '25 14:03 torarnv

I can't reproduce :'(

 ✔ Container truc-caddy-reproducer-1  Created                              0.1s 
        ⦿ Watch enabled
Attaching to caddy-reproducer-1
caddy-reproducer-1  | {"level":"info","ts":1742208021.9712799,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-reproducer-1  | {"level":"info","ts":1742208021.9721558,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-reproducer-1  | {"level":"info","ts":1742208021.9731705,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-reproducer-1  | {"level":"warn","ts":1742208021.97328,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
caddy-reproducer-1  | {"level":"info","ts":1742208021.9734535,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x400049c780"}
caddy-reproducer-1  | {"level":"warn","ts":1742208021.9735224,"logger":"http","msg":"HTTP/2 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1  | {"level":"warn","ts":1742208021.9735262,"logger":"http","msg":"HTTP/3 skipped because it requires TLS","network":"tcp","addr":":80"}
caddy-reproducer-1  | {"level":"info","ts":1742208021.9735277,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
caddy-reproducer-1  | {"level":"info","ts":1742208021.9772449,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
caddy-reproducer-1  | {"level":"info","ts":1742208021.9772654,"msg":"serving initial configuration"}
caddy-reproducer-1  | {"level":"info","ts":1742208021.9792316,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
caddy-reproducer-1  | {"level":"info","ts":1742208021.980091,"logger":"tls","msg":"finished cleaning storage units"}
                    ⦿ Syncing service "caddy-reproducer" after 1 changes were detected
caddy-reproducer-1  | {"level":"info","ts":1742208586.0151498,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
caddy-reproducer-1  | {"level":"warn","ts":1742208586.0156841,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
caddy-reproducer-1  | {"level":"info","ts":1742208586.0158362,"logger":"http","msg":"servers shutting down with eternal grace period"}
caddy-reproducer-1  | {"level":"info","ts":1742208586.017201,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
caddy-reproducer-1  | {"level":"info","ts":1742208586.017228,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
                    ⦿ service(s) ["caddy-reproducer"] restarted
caddy-reproducer-1 exited with code 0
caddy-reproducer-1  | {"level":"info","ts":1742208586.492455,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
caddy-reproducer-1  | {"level":"info","ts":1742208586.4934077,"msg":"adapted config to JSON","adapter":"caddyfile"}
caddy-reproducer-1  | {"level":"info","ts":1742208586.494336,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//localhost:2019","//[::1]:2019","//127.0.0.1:2019"]}
caddy-reproducer-1  | {"level":"warn","ts":1742208586.4945173,"logger":"http.auto_https","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}

ndeloof avatar Mar 17 '25 10:03 ndeloof

Strange :) In my case I'm running docker compose up --watch on my laptop, but with the docker context pointing to my NAS. Could that affect this somehow?

torarnv avatar Mar 17 '25 10:03 torarnv

Hello @torarnv I tried to reproduce your issue, in between we released a signifiant number of Compose release and I'm not able to reproduce your issue

caddy-reproducer-1  | {"level":"info","ts":1744638529.4268618,"logger":"tls","msg":"finished cleaning storage units"}
                    ⦿ Syncing service "caddy-reproducer" after 1 changes were detected
caddy-reproducer-1  | {"level":"info","ts":1744638567.5292418,"msg":"shutting down apps, then terminating","signal":"SIGTERM"}
caddy-reproducer-1  | {"level":"warn","ts":1744638567.529311,"msg":"exiting; byeee!! 👋","signal":"SIGTERM"}
caddy-reproducer-1  | {"level":"info","ts":1744638567.5293336,"logger":"http","msg":"servers shutting down with eternal grace period"}
caddy-reproducer-1  | {"level":"info","ts":1744638567.5295353,"logger":"admin","msg":"stopped previous server","address":"localhost:2019"}
caddy-reproducer-1  | {"level":"info","ts":1744638567.529549,"msg":"shutdown complete","signal":"SIGTERM","exit_code":0}
                    ⦿ service(s) ["caddy-reproducer"] restarted
caddy-reproducer-1 exited with code 0
caddy-reproducer-1  | {"level":"info","ts":1744638567.8666615,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}

Can you let me know if the latest version of Compose fixed the issue also for you?

glours avatar Apr 14 '25 13:04 glours

Not OP, but I also had this issue, just upgraded from v2.28 to v2.38 and it fixed the issue. docker compose up will stay connected after watch restarts the image.

EricChen1248 avatar Aug 15 '25 13:08 EricChen1248

I have version 2.39.2 and I still have the issue

bashirmindee avatar Sep 16 '25 09:09 bashirmindee