Compose config with --no-interpolate fails as it tries to validate environment variables as ports

[padraic-padraic]

Description of the issue

When trying combine compose and override files with docker-compose config --no-interpolate, I get an validation error in the ports block for a service that uses environment variables.

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.25.0, build unknown

Output of docker version

Docker version 19.03.8, build afacb8b7f0

Output of docker-compose config (Make sure to add the relevant -f and other flags) The issue relates to config!

Steps to reproduce the issue

1. Small example file

    #compose.yml
    version: '3'
    services:
       test:
         image: hello-world
         ports:
             - "${DUMMY_PORT}:${DUMMY_PORT}"
   

2. docker-compose -f compose.yml config --no-interpolate

Observed result

services.test.ports is invalid: Invalid port "${DUMMY_PORT}:${DUMMY_PORT}", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]

Expected result

version: '3'
services:
  test:
    image: hello-world
    ports:
      - "${DUMMY_PORT}:${DUMMY_PORT}"

Stacktrace / full error message

compose.config.config.find: Using configuration files: ./compose.yml
ERROR: compose.cli.main.main: The Compose file './compose.yml' is invalid because:
services.test.ports is invalid: Invalid port "${DUMMY_PORT}:${DUMMY_PORT}", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]

Additional information

OS version / distribution, docker-compose install method, etc. Pop! OS 20.04, using system packages for docker / docker-compose.

[o-sokolova]

How detailed should docker-compose validate variables in port field? The whole pattern is [[remote_ip:]remote_port[-remote_port]:]port[-port][/protocol] I think it would be redundant to validate case when variables are in every possible place. How you think? Finally, should we check format of declaration like "${VAR:-1234}"?

[emersonf]

Any bandwidth to make progress on this, e.g. PR feedback?

[pseusys]

We really need this bug fixed! What's the progress?

[ndeloof]

Compose v1 development is now paused as Compose v2 has been release. I don't expect some significant progress made for this issue regarding the python codebase.

[pseusys]

It is still an issue for Docker Compose v2, however the error message changed: For code line ports: ${FIRST_VARIABLE}:${SECOND_VARIABLE} it produces error: error decoding 'Ports': Invalid containerPort: ${SECOND_VARIABLE}. Docker compose version 2.2.3.

[pseusys]

Update: error appears for every variable in ports section, equally in cases: - ${VARIABLE}, - 42:${VARIABLE}, - ${VARIABLE}:42 and ${FIRST_VARIABLE}:${SECOND_VARIABLE}

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it had not recent activity during the stale period.

[Phlegethon90]

Any progress?

[stale[bot]]

This issue has been automatically marked as not stale anymore due to the recent activity.

[ndeloof]

re-opening issue as this also applies to compose v2

[bintoll]

I also face this issue

[ndeloof]

The root cause here is that compose-go maps compose.yaml to go structs, with typed attributes. So it needs to convert input into int and can't manage a yaml input with non-interpolated entries. Re-designing compose-go to offer the same features (i.e merge yaml file and validate schema) without go bindings will be challenging.

[dmarteau]

You may use the long syntax as a workaround: https://docs.docker.com/compose/compose-file/compose-file-v3/#long-syntax-1

[ndeloof]

compose-go/v2 is currently in development and will allow to process compose files as plain yaml trees, without relying on typed go structs. This will allow to validate a compose file with interpolation disabled

[ecmonsen]

I'd love to see this fixed. Is there an ETA for compose-go/v2?

[ndeloof]

While compose-go/v2 is a first place toward a parser which can process compose.yaml with non-interpolated values, this is not ready yet so it can fully implement docker compose config feature. no ETA as this is a long-running refactoring process

[milas]

This fix has been included in Compose 2.24.7+. If you continue to have problems after upgrading, comment here and we can re-open the issue or create a new one as appropriate. Thanks for the report!