Newly started renamed service in detached mode does not attach its output to already started live log feed from docker-compose up

[devision-milen-dzhambazov]

Description

Steps to reproduce the issue:

1. Run docker-compose up
2. In another terminal run docker-compose stop <service>, rename the service in docker-compose.yml and then run docker-compose up -d <new_service_name>

Describe the results you received: The renamed service's logs do not get displayed in the terminal which runs the docker-compose stack in foreground.

Describe the results you expected: The terminal which displays live feed of the logs of the docker-compose stack should attach to the newly started renamed service and display its logs as well.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker compose version:

Docker Compose version v2.10.2
docker-compose version 1.25.0, build unknown

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.10.2)
  extension: Manages Docker extensions (Docker Inc., v0.2.9)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.19.0)

Server:
 Containers: 19
  Running: 19
  Paused: 0
  Stopped: 0
 Images: 533
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.16.3-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 27.42GiB
 Name: docker-desktop
 ID: 5H5V:5P7J:5VN3:2PKX:VFGH:72CC:CH6H:Y5CB:RWO5:IGSN:CC43:XD5G
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional environment details:

[milas]

This behavior is more or less "working as designed" / expected with Compose's current design. In general, if you're editing the compose.yaml file or starting/stopping things, you'll generally need to restart any log tails.

Is there a specific use case you're trying to accomplish where you're hitting this?

[devision-milen-dzhambazov]

No specific use case. Just thought it would be the expected behavior. I don't see a reason to not attach to the logs of the same stack despite the service being renamed.

[nicksieger]

Yeah, what's happening is the first docker compose up creates and attaches to the containers for the services defined in compose.yaml at the time the command was run. Your additional commands create new containers that the original command isn't attached to. Like @milas said, compose doesn't watch for any changes or new containers started since it launched. Not saying it couldn't be made to do so, but that would be more of a feature request. If you're interested in seeing that behavior, feel free to re-open or open a new issue describing what you'd like to see. Thanks for filing!

[devision-milen-dzhambazov]

I'm not sure how to reopen the ticket. I think it's more intuitive if the docker-compose up creates some type of hook/handle with a unique identifier based on the directory where the docker-compose.yml file is, which can be used to attach new services to its logs. Because I assumed that this is the default behavior I was thrown off when I wasn't seeing the logs of newly added services to the already running stack. I went as far as to diagnose the docker and service's logging. The expected behavior IMO would be to attach any newly added services logs to the already running foreground docker-compose processes.

I guess another feature request within this problem domain would be the fact that when you run docker-compose up in another console on a docker-compose stack that is already running, it would list all of the logs (sometimes taking hours until it gets to the tail), which were ever produced by running this stack over all of its runs over time instead of just displaying the logs of the current run, as it does in the console, which started the stack last.