compose
compose copied to clipboard
docker
[BUG] docker compose up --watch with sync+restart does not show logs after restart
Description
When running docker compose up --watch, the logs for the service appear in the output.
When specifying sync+restart as the action on a file change, when a file is changed, the container restarts, but there are no more logs outputted. When running docker compose logs, you can see the app was restarted correctly and is still logging.
This behaviour does not happen when using rebuild as the action (although the logs aren't picked up straight away so the first logs are potentially missed).
Expected behaviour is that when using sync+restart, when a file is changed, the logs from the container continue to be shown in the output of docker compose up --watch.
Steps To Reproduce
Dockerfile:
FROM node:slim
COPY src src
Contents of src can be anything.
Docker Compose:
services:
app:
build: .
command: >-
bash -ceuo pipefail "
echo restart
i=0
while true; do echo boop $$i && ((i++)) && sleep 1; done
"
stop_signal: SIGKILL
develop:
watch:
- action: sync+restart
path: ./src
target: /src
Running docker compose up --watch, then saving a file in src outputs:
➜ docker compose up --watch
[+] Running 1/2
✔ Network docker-compose-watch_default Created 0.1s
⠇ Container docker-compose-watch-app-1 Created 0.8s
⦿ Watch enabled
Attaching to app-1
app-1 | restart
app-1 | boop 0
app-1 | boop 1
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
app-1 | boop 6
⦿ Syncing "app" after changes were detected
[+] Restarting 1/1
✔ Container docker-compose-watch-app-1 Started 0.6s
app-1 exited with code 0
When running docker compose logs, it shows all the logs correctly.
➜ docker compose logs
app-1 | restart
app-1 | boop 0
app-1 | boop 1
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
app-1 | boop 6
app-1 | restart
app-1 | boop 0
app-1 | boop 1
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
If you change sync+restart to rebuild in the docker-compose file, and run docker compose up --watch, and save a file, it now prints (most of) the logs correctly:
➜ docker compose up --watch
[+] Running 1/2
✔ Network docker-compose-watch_default Created 0.1s
⠧ Container docker-compose-watch-app-1 Created 0.7s
⦿ Watch enabled
Attaching to app-1
app-1 | restart
app-1 | boop 0
app-1 | boop 1
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
⦿ Rebuilding service "app" after changes were detected...
app-1 | boop 6
⦿ service "app" successfully built
app-1 exited with code 0
app-1 has been recreated
app-1 | boop 2
app-1 | boop 3
app-1 | boop 4
app-1 | boop 5
app-1 | boop 6
Note that in the output, it misses the first few logs.
Doing a docker compose logs only shows the logs for the latest container.
Compose Version
➜ docker compose version
Docker Compose version v2.26.1
```sh
➜ docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019
### Docker Environment
```Text
➜ docker info
Client: Docker Engine - Community
Version: 26.1.0
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.14.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.26.1
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 125
Server Version: 26.1.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: e377cd56a71523140ca6ae87e30244719194a521
runc version: v1.1.12-0-g51d5e94
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.5.0-27-generic
Operating System: Ubuntu 22.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.31GiB
Name: --
ID: QBED:UL4T:4VZM:D6KX:NMXE:WYFE:6DDN:PK4Y:CEGA:C5IZ:WBPG:JZES
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Anything else?
Similar behavior happens when sending a docker compose up --watch job to the background and watching the output with docker compose logs --follow in another command window. The service logs show up until the service restart occurs, after which the logs command exits with code 0. One might expect docker compose logs --follow to show the sync/restart messages along with the rest of the output from the restarted (or terminated and started again) service.
I think this happens when the number of running services drops to zero, even if temporarily for a restart. For example, if you start two services but one exits due to an error, this issue will occur when the other one is restarted.
I was able to work around this issue for now by keeping a dummy service alive at all times:
dummy-watch-workaround:
image: alpine:latest
init: true
command: ["sh", "-c", "while true; do sleep 2; done"]
@dkaste I confirm. Let's say I have a db and a web service, the web service being the one I want to watch in sync+restart mode.
Just running docker compose up --watch web ends up with the terminal detaching from the service with web-1 exited with code 0.
But running docker compose up --watch db web behaves correctly.
