cli
cli copied to clipboard
docker
Docker Stack Deploy error when using `depends_on` long syntax
Description
Given the following dummy docker-compose.yml
version: '3.9'
services:
serviceA:
image: serviceA:version1
serviceB:
image: serviceB:version2
depends_on:
serviceA:
condition: service_completed_successfully
where serviceB finishes once run (in my real world example, it's a container that runs migrations on a database then dies)
The docker compose up command doesn't complain, however, the docker stack deploy command complains by saying that
services.serviceB.depends_on must be a list
I've tried running docker compose up and it doesn't complain about that, so I assume it works for docker compose up but not for docker stack deploy
Documentation regarding long syntax of depends_on can be found here
https://docs.docker.com/compose/compose-file/#long-syntax-1
Reproduce
- Create the
docker-compose.ymlfile I showed - Execute
docker stack deploy --compose-file docker-compose.yml example-stack
Expected behavior
The stack should start creating. With the example file I provided, it should complain that image serviceA doesn't exist because it's an example, but with real world images it should work and start the stack. However, what actually ends up happening is the error i stated above about the depends on not being a list.
docker version
Client: Docker Engine - Community
Version: 20.10.9
API version: 1.41
Go version: go1.16.8
Git commit: c2ea9bc
Built: Mon Oct 4 16:08:29 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.9
API version: 1.41 (minimum version 1.12)
Go version: go1.16.8
Git commit: 79ea9d3
Built: Mon Oct 4 16:06:37 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.8
GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
compose: Docker Compose (Docker Inc., v2.6.0)
scan: Docker Scan (Docker Inc., v0.9.0)
Server:
Containers: 3
Running: 1
Paused: 0
Stopped: 2
Images: 20
Server Version: 20.10.9
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: olm6j3as729ahukyet1j199mj
Is Manager: true
ClusterID: oj0ovckoc67ev73rqj093a052
Managers: 1
Nodes: 1
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 10.0.0.76
Manager Addresses:
10.0.0.76:2377
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc version: v1.1.4-0-g5fd4c4d
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.4.0-132-generic
Operating System: Linux Mint 20.3
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 31.29GiB
Name: worklaptop
ID: OVQD:AGQF:NNW2:HPOW:4DNN:CDIK:JKYR:4DSN:YDJX:7XFJ:XJSR:JTXB
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit support
Additional Info
No response
Any update or thoughts from the developers on this?
Not that depends_on is expected to actually be honored by stack deploy, but this issue means there is a format incompatibility which prevents the same file being used to deploy using either compose up (on a single host) or stack deploy (on a swarm) - which is a very nice feature from an environment flexibility standpoint. While this bug exists, you'd need to maintain two separate files for this use-case, which is not ideal for obvious reasons.
+1
Guys, what is the status of the task? Need an implementation, how to push a task?
There is a bug that has remained unfixed for approximately six months. It may be possible to use GPT-4 to write a PR to address the issue, as it has not received attention from officials.
I run into this today and only solution I can see is using restart_policy with condition and delay properties.
The Swarm's development is pretty slow so I don't expect this resolved soon.
Anyone using "docker compose -f fileA.yaml -f fileB.yaml config -o merged.yaml" will run into this issue when trying to deploy on swarm with "docker stack deploy -c merged.yaml stack". Ran into this issue when trying to use github action for stack deploy with multiple files (with https://github.com/kitconcept/docker-stack-deploy)
I'm facing the same problem. I transitioned from the discussion at https://github.com/moby/moby/issues/29133#issuecomment-442912378, then moved on to https://github.com/docker/compose/issues/9306#issuecomment-1464027534, and finally arrived at this current problem.
hello there!
there seems to be still an issue with depends_on when using docker compose config to pipe a merged compose model into docker stack deploy, see:
❯ docker compose -f compose.production.yaml config | docker stack deploy prod -c -
services.cli.depends_on must be a list
❯ cat compose.production.yaml
include:
- path:
- .docker/base.yaml
- .docker/db.yaml
❯ grep "depends_on" .docker/base.yaml -A3
depends_on:
- caddy
- redis
environment:
❯ docker compose -f compose.production.yaml config | grep "depends_on" -A7
depends_on:
caddy:
condition: service_started
required: true
redis:
condition: service_started
required: true
deploy:
Why do I need this? What am I trying to do?
I'd like to split complex compose files by using one of the latest compose features: include.
Sadly, I get an error while trying to put include in my compose.production.yaml while using docker stack deploy, see:
❯ docker stack deploy prod -c compose.production.yaml
(root) Additional property include is not allowed
So this is why I tried to use docker compose config to resolve the include but this leads to other syntax errors, as seen in the first paragraph.
At this point I have some questions and hope that someone here can provide answers:
1. Why does docker stack deploy not support include?
2. Why is docker stack deploy not able to handle the syntax generated by docker compose config?
3. What are the main implementation issues which lead to the inconsistent support between compose up and stack deploy?
4. Does it not say -c, --compose-file strings Path to a Compose file which implicates (full) compose spec support?
5. Am I wrong thinking that the docker-cli implements both of the named subcommands and should be able to support the same (latest) compose application model?
6. Has Docker Inc. decided to abandon Docker Swarm at some point? Did I miss that and do I have to let go in favour of k8s?
7. Is there no need to give the community a feature like kustomize to split and better reuse different deployment parts (include seemed to be a candidate for this) to give the possibility to stick with swarm clusters (as there are cases where this is perfectly fine)?
PS: I use Docker since 2015/2016 and it became a key figure in my toolbox. My journey with docker, docker compose and docker swarm was a lot of joy but recently and especially with complex projects it became also very frustrating as the tool one loved so much seems to truly rust.
PPS: If there is a better place to put the above, please let me know. I will not hesitate to pass on my words to the right recipient.
I have spoken.
@jawys
docker compose is not a docker stack pre-processor. The yaml output it produces by docker compose config is compliant with the compose specification, while docker stack isn't. Nothing we can do here until stack adopts the compose spec.
there seems to be still an issue with depends_on
docker stack does not support depends_on, this is the reason this was removed from compose file format version: "3" before compose specfication made this obsolete.
Why does docker stack deploy not support include?
docker swarm and stack subcommand is owned by Mirantis, so I can't tell why there's no move with this topic
Why is docker stack deploy not able to handle the syntax generated by docker compose config?
the parser being used by this command was used to produce compose-spec reference implementation, but evolved a lot in the meantime. Typically with introduction of imports. stack still uses the legacy codebase.
Does it not say -c, --compose-file strings Path to a Compose file which implicates (full) compose spec support?
Nope. It just involves "docker compose" file as set by version: "3.9" before the compose spec was created
Am I wrong thinking that the docker-cli implements both of the named subcommands and should be able to support the same (latest) compose application model?
That's terribly confusing, but that's our legacy. Things are even more insane as docker compose is technically a dedicated binary which can easily get updated while docker stack is implemented in docker/cli which make it non-trivial to guess who is in charge for maintenance. I suggested to create a dedicated swarm CLI to address
this, but I'm not the one to take the final decision here :)
Has Docker Inc. decided to abandon Docker Swarm at some point?
Docker Inc does not own Docker Swarm anymore. This is a Mirantis product since https://www.mirantis.com/blog/mirantis-acquires-docker-enterprise-platform-business/
@ndeloof thank you very much for taking the time to give an adequate explanation to my questions. Appreciate it a lot <3
https://docs.docker.com/compose/compose-file/compose-file-v3/#depends_on
So for now, this configuration will not take effect anyway, right?
