[BUG] docker compose v2.24.0 complains about root properties in 'extends' template file

[mauke]

Description

Docker compose has suddenly started complaining about unknown top-level properties in configuration templates that are only used via extends, which is breaking my builds. It didn't used to.

Steps To Reproduce

Create file a:

name: tmp

services:
  s0:
    extends:
      file: b
      service: s-base

Create file b:

"(this is file is meant to be used via 'extends' only; use 'docker compose -f a' instead!)": null

services:
  s-base:
    image: busybox

Then run:

$ docker run -v ".:/mnt:ro" --rm docker:24.0.7-cli docker compose -f /mnt/a config
validating /mnt/b: (root) Additional property (this is file is meant to be used via 'extends' only; use 'docker compose -f a' instead!) is not allowed

Expected behavior:

$ docker run -v ".:/mnt:ro" --rm docker:23-cli docker compose -f /mnt/a config
name: tmp
services:
  s0:
    image: busybox
    networks:
      default: null
networks:
  default:
    name: tmp_default

Compose Version

Docker Compose version v2.24.0

Docker Environment

No response

Anything else?

No response

[jhrotko]

Hello, @mauke

I am not sure what you are trying to achieve with the first line in b -> "(this is file is meant to be used via 'extends' only; use 'docker compose -f a' instead!)": null. Could you give more details?

The error message you are receiving is complaining that it does not recognise (this is file is meant to be used via 'extends' only; use 'docker compose -f a' instead!) as a valid field.

According to the Extension docker compose docs section the only exception for unrecognised fields are fileds starting with x-{YOUR_FIELD}

[mauke]

I'm trying to prevent the accidental use of b as the main/only compose config. It is purely a template file that only contains partial service fragments.

It used to be that docker compose -f b would show the error message above, but docker compose -f a would silently ignore the property and work as intended.

Now docker compose -f a also shows the error. Is there another way to make sure a config file is only used via extends, not directly?

[jhrotko]

@mauke so, you would like b to be somewhat abstract and not possible for its services to be instantiated (ever)?

[mauke]

Yes, exactly.

[ndeloof]

extends can refer to an external compose file to reuse a service definition, but (whenever this was loosely checked so far) such a file MUST be a valid compose file, so you CAN'T introduce unsupported attribute this way.

There's no such concept as an "abstract" service definition in compose model, could be an interesting feature, but then need an explicit syntax.

[jhrotko]

Hello, @mauke!

I would like to better understand your real use case to make sure we could add this as a feature to compose. Could you provide more details in the real scenario you are using this?

[mauke]

The context is a project with several services, many of which have similar settings. For example, all services have restart: always. Other fields that repeat a lot are volumes and environment. Instead of copy/pasting the same information in every service information, I tried to factor out the common settings for each service group and put them in an "abstract base service" in a separate file (a "settings library").

That's how you get a structure like this:

compose.common.yaml:

services:
  abstract-base:
    restart: always
    volumes:
      /etc/localtime:/etc/localtime:ro

  abstract-frobnicate:
    extends: abstract-base
    volumes:
      /data/frobnicate:/data
    environment:
      CONFIG: /data/config

  abstract-twiddle:
    extends: abstract-base
    volumes:
      /data/twiddle:/data
      /etc/twiddle:/etc/twiddle:ro
    environment:
      AUTOCLEAN: 1
      CACHE: /data/cache

compose.production.yaml:

services:
  service-A:
    extends:
      file: compose.common.yaml
      service: abstract-frobnicate
    image: image-A
    environment:
      BOROGOVE: mimsy

  service-B:
    extends:
      file: compose.common.yaml
      service: abstract-frobnicate
    image: image-B

  service-C:
    extends:
      file: compose.common.yaml
      service: abstract-twiddle
    image: image-C

  service-D:
    extends:
      file: compose.common.yaml
      service: abstract-twiddle
    image: image-D
    volumes:
      /etc/maxwelld:/etc/maxwelld:ro

Now, since compose.common.yaml is intended to be used strictly as a library of abstract base services, in order to avoid accidents, I was looking for a way to make docker compose -f compose.production.yaml work, but docker compose -f compose.common.yaml fail with a reasonably comprehensible error message. Adding a top-level property with a speaking name provided exactly that.

As for

There's no such concept as an "abstract" service definition in compose model

The documentation strongly implies there is:

With extends you can define a common set of service options in one place and refer to it from anywhere.

Note that it does not say "a service"; it says "a common set of service options". Which is exactly what I mean by "an abstract base service".

[ndeloof]

There's no way (yet) to get compose file display such a warning message as a compose file is "not intended to be used directly", but you can make it invalid, for example if your service has no image nor build section set, so that one can't directly use it:

services:
  do-not-use-this-compose-file: {}
    # no image nor build section

  common:
    environment:
      FOO: BAR

$ docker compose -f c.yaml config
service "do-not-use-this-compose-file" has neither an image nor a build context specified: invalid compose project

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.