docker
Flag to indicate a service completes for `docker compose up --wait`
Description
Say you have a docker compose file that has 2 services:
- Stand up a database container
- Run migrations against that database container
services:
postgres:
image: postgres:latest
environment:
POSTGRES_USER: leo
POSTGRES_PASSWORD: 123
POSTGRES_DB: db
ports:
- "5432:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready"]
interval: 1s
postgres-migrations:
build:
context: .
dockerfile: Dockerfile-goose
volumes:
- ./migrations:/migrations
depends_on:
postgres:
condition: service_healthy
command: >
sh -c " goose -dir /migrations postgres 'host=postgres user=leo password=123 dbname=db' up && touch /tmp/done &&
sleep 2"
healthcheck:
test: ["CMD", "test", "-f", "/tmp/done"]
interval: 1s
And a test script to run some integration tests after the migrations have been run on the db:
#!/usr/bin/env sh
set -e
docker compose up --wait
echo "docker compose up complete"
# the "integration tests"
PGPASSWORD=123 psql -h localhost -p 5432 -U leo -d db -c "SELECT * FROM movies" || echo "failed"
# clean up
docker compose down
Dockerfile-goose
FROM golang:alpine as builder
RUN apk add --no-cache git
RUN go install github.com/pressly/goose/v3/cmd/goose@latest
migrations/20240221040043_run.sql
-- +goose Up
-- +goose StatementBegin
SELECT pg_sleep(2); -- simulate migrations taking longer than they do
CREATE TABLE movies (
id SERIAL PRIMARY KEY,
title VARCHAR(255) NOT NULL
);
INSERT INTO movies (title) VALUES ('Woohoo');
-- +goose StatementEnd
-- +goose Down
-- +goose StatementBegin
DROP TABLE IF EXISTS movies;
-- +goose StatementEnd
Notice how the docker compose file has to do some acrobatics with a "done" file flag. This is because docker compose up --wait does the following: Wait for services to be running|healthy. Implies detached mode.
The done file flag is used to make the migrations service only report healthy when the migrations are complete, so up --wait blocks until then and exits successfully so the test script continues.
Feature Request
It would be nice if you could indicate to up --wait that certain services will complete and up --wait should exit successfully if they do so, maybe with a new annotation in the compose file like:
services:
postgres-migrations:
will_complete: true
That way the test script can remain agnostic to the names of particular services or containers and the compose file doesn't have to implement the "done" file creation and healthcheck.
I've found you can do something like this
my_service:
restart: "no"
healthcheck:
# interval etc. is inherited via `extends`
test: "false"
Since the container will exit upon completion, the zero exit code is sufficient for Docker Compose to go "yep, that's done".
@ShadowLNC can you modify my example with your suggestion so that it achieves my stated goal? I haven't been able to do so myself
@robinovitch61 I think this should work:
docker-compose.yml
services:
postgres:
image: postgres:latest
environment:
POSTGRES_USER: leo
POSTGRES_PASSWORD: 123
POSTGRES_DB: db
ports:
- "5432:5432"
healthcheck:
test: ["CMD-SHELL", "pg_isready"]
interval: 1s
postgres-migrations:
build:
context: .
dockerfile: Dockerfile-goose
volumes:
- ./migrations:/migrations
depends_on:
postgres:
condition: service_healthy
# Changes begin here
command: goose -dir /migrations postgres 'host=postgres user=leo password=123 dbname=db' up
restart: "no"
healthcheck:
test: "false" # or ["CMD-SHELL", "false"]
interval: 1s
Looking at the exit code, it seems that even if the migrations container exits 0, docker compose up --wait will exit 1 if any container stopped.
I tried the following:
# Probably don't need the second postgres-migrations argument, just up everything in the file
docker compose up --abort-on-container-failure --exit-code-from postgres-migrations postgres-migrations
This seemed to work. I thought it might quit postgres, since the abort flage specifies incompatibility with -d or --wait, but the postgres container also stayed running for me even when I used --abort-on-container-exit. (I do have other services depending on postgres, but I specifically took them down, so you could add a dummy service which is never started, if this turns out to be necessary.)
Thanks for the example. A critical part for me is:
That way the test script can remain agnostic to the names of particular services
So while your edit works when I edit the test script to include a reference to the postgres-migrations service explicitly, it doesn't satisfy this request for me.
Ahh, I missed that, apologies. You could add a secondary service, something like
services:
# keep postgres and postgres-migrations from previous example
migrations-done:
image: alpine:latest
depends_on:
postgres-migrations:
condition: service_completed_successfully
command: sleep infinity
stop_signal: SIGKILL # Doesn't respond to SIGTERM
healthcheck:
test: "true"
interval: 1s
This would take the logic out of the migrations container while allowing you not to specify the container name, but it's still a little verbose.
Amusingly enough, I just ran into the exact same issue when using profiles - depending on the profile set, sometimes the last service in a dependency chain is a "oneshot" (migrations) container and of course compose up --wait exits 1, so my script thinks that Compose is reporting an error.
Obviously I can't know ahead of time what profiles the user will set, so I'm considering a wrapper script to run compose config and then parse the output and look for things which have no dependent containers, and compose up that last service separately if (and only if) it has my healthcheck: false marker. I know Compose also behaves differently if the last service has no healthcheck, but it won't wait for completion, only for it to start, which doesn't really satisfy this use case.