[BUG] Inconsistent behavior of profiles and remove-orphans

[albanf]

trafficstars

Description

I would expect services not activated by profiles to be considered orphans, since they don't appear in the output of the config command.

Current behavior:

• docker compose up --remove-orphans: services in inactive profiles are not removed
• docker compose down --remove-orphans: services in inactive profiles are not removed
• docker compose down --project-name <name> (whether --remove-orphans is specified or not): services in inactive profiles are removed

Expected behavior:

• docker compose up --remove-orphans: services in inactive profiles are removed
• docker compose down: services in inactive profiles are not removed
• docker compose down --remove-orphans: services in inactive profiles are removed And it should not depend on the --project-name option

Steps To Reproduce

docker-compose.yml:

services:
  web1:
    image: nginx
  web2:
    image: nginx
    profiles: ["web2"]      

docker-compose-extra.yml:

services:
  web3:
    image: nginx

$ COMPOSE_FILE=docker-compose.yml:docker-compose-extra.yml COMPOSE_PROJECT_NAME=dcprofiles COMPOSE_PROFILES=web2 docker compose up -d
[+] Running 4/4
 ✔ Network dcprofiles_default   Created
 ✔ Container dcprofiles-web3-1  Started
 ✔ Container dcprofiles-web1-1  Started
 ✔ Container dcprofiles-web2-1  Started

$ COMPOSE_PROJECT_NAME=dcprofiles docker compose config
name: dcprofiles
services:
  web1:
    image: nginx
    networks:
      default: null
networks:
  default:
    name: dcprofiles_default

$ COMPOSE_PROJECT_NAME=dcprofiles docker compose down
[+] Running 2/1
 ✔ Container dcprofiles-web1-1  Removed
 ! Network dcprofiles_default   Resource is still in use

=> only web1 is removed

$ COMPOSE_PROJECT_NAME=dcprofiles docker compose down --remove-orphans
[+] Running 2/1
 ✔ Container dcprofiles-web3-1  Removed
 ! Network dcprofiles_default   Resource is still in use

=> web3 is removed but not web2

$ COMPOSE_PROJECT_NAME=dcprofiles docker compose --project-name dcprofiles down
[+] Running 2/2
 ✔ Container dcprofiles-web2-1  Removed
 ✔ Network dcprofiles_default   Removed

=> web2 is also removed, no need to specify --remove-orphans

With up the behavior is:

$ COMPOSE_FILE=docker-compose.yml:docker-compose-extra.yml COMPOSE_PROJECT_NAME=dcprofiles COMPOSE_PROFILES=web2 docker compose up -d
[+] Running 4/4
 ✔ Network dcprofiles_default   Created
 ✔ Container dcprofiles-web3-1  Started
 ✔ Container dcprofiles-web1-1  Started
 ✔ Container dcprofiles-web2-1  Started

$ COMPOSE_PROJECT_NAME=dcprofiles docker compose up -d
WARN[0000] Found orphan containers ([dcprofiles-web3-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. 
[+] Running 1/0
 ✔ Container dcprofiles-web1-1  Running

=> no container is removed

$ COMPOSE_PROJECT_NAME=dcprofiles docker compose up -d --remove-orphans
[+] Running 2/1
 ✔ Container dcprofiles-web3-1  Removed                                                                                                                                                                                                                                              0.4s 
 ✔ Container dcprofiles-web1-1  Running

=> web3 is removed, but not web2

$ COMPOSE_PROJECT_NAME=dcprofiles docker compose --project-name dcprofiles up -d --remove-orphans
[+] Running 1/0
 ✔ Container dcprofiles-web1-1  Running

=> no special effect of --project-name

Compose Version

$ docker compose version
Docker Compose version v2.27.0

Docker Environment

Client:
 Version:    24.0.5
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.0
    Path:     /home/user/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 8
  Running: 6
  Paused: 0
  Stopped: 2
 Images: 120
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 
 runc version: 
 init version: 
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-28-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 32
 Total Memory: 62.48GiB
 Name: xxxxxxxxx
 ID: xxxxxx
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/16, Size: 24

Anything else?

No response

[ndeloof]

I would expect services not activated by profiles to be considered orphans

Those are not. They are well defined in the compose model, but disabled. Some compose users rely on this feature to run partial application shut down.

If you want to stop all services, you can run command with --profile='*', or pass project name so compose file is not even parsed and all containers matching the project label will be removed

[jhrotko]

full command: COMPOSE_PROJECT_NAME=dcprofiles docker compose --profile='*' down --remove-orphans

[elderapo]

Right now it's pretty hard to elegantly sync which services are supposed to be up or down by just changing COMPOSE_PROFILES. There is no elegant solution to bring down services whose profile is no longer specified in COMPOSE_PROFILES. Universal command docker compose --profile=* down && docker compose up -d does the job but unnecessarily restarts all other containers.

I propose --remove-inactive-profiles option for docker compose up/down with or without --remove-oprhans. Alternatively --inactive-profiles-as-orphans would do the job as well (ex. docker compose --inactive-profiles-as-orphans --remove-orphans up -d) but won't cover an edge-case where the user might want to remove services no longer specified in profiles but not services that were removed from docker-compose.yml file.

[blop]

In the same kind of issue, when you update the "profiles" of a service, it docker compose will recreate the service even if nothing else has been modified. For example if you just add or remove another extra profile from the list.

[ndeloof]

@blop nice catch. profile should not be included in service hash so no such recreate takes place, could you please create a dedicated issue ?

[blop]

Done #12260 :)

[hh10k]

docker compose config doesn't include services from inactive profiles, so if you run docker compose config > tmp.yml && docker compose -f tmp.yml up --remove-orphans you will get different behaviour from docker compose up --remove-orphans.

Is this expected behaviour of docker compose config?

[ndeloof]

@hh10k yes it is. Running command with && you truncate model from inactive profiles, so the result isn't the same. docker compose config role is not to keep the model unchanged