docker
Docker Compose does not support defining port ranges in long form.
Issue
docker-compose file format does not support a method of declaring port ranges in the long form syntax defined here: https://docs.docker.com/compose/compose-file/#long-syntax-1
Steps to Reproduce
docker-compose.yml:
version: '3.4'
services:
helloWorld:
image: hello-world
ports:
- target: 9900-9999
published: 9900-9999
protocol: tcp
mode: host
command executed:
docker-compose -f ./docker-compose.yml config
docker-comopse -f ./docker-compose.yml up -d
Expected Output:
PS D:\issue> docker-compose -f ./docker-compose.yml config
services:
helloWorld:
image: hello-world
ports:
- target: 9900-9999
published: 9900-9999
protocol: tcp
mode: host
version: '3.4'
PS D:\issue> docker-compose -f ./docker-compose.yml up -d
<ContainerId>
Actual Output:
PS D:\issue> docker-compose -f ./docker-compose.yml config
Traceback (most recent call last):
File "docker-compose", line 6, in <module>
File "compose\cli\main.py", line 71, in main
File "compose\cli\main.py", line 118, in perform_command
File "compose\cli\main.py", line 304, in config
File "compose\cli\command.py", line 47, in get_config_from_options
File "compose\config\config.py", line 375, in load
File "compose\config\config.py", line 506, in process_config_file
File "compose\config\config.py", line 497, in interpolate_config_section
File "compose\config\interpolation.py", line 44, in interpolate_environment_variables
File "compose\config\interpolation.py", line 44, in <genexpr>
File "compose\config\interpolation.py", line 39, in process_item
File "compose\config\interpolation.py", line 39, in <genexpr>
File "compose\config\interpolation.py", line 54, in interpolate_value
File "compose\config\interpolation.py", line 77, in recursive_interpolate
File "compose\config\interpolation.py", line 74, in recursive_interpolate
File "compose\config\interpolation.py", line 74, in <genexpr>
File "compose\config\interpolation.py", line 70, in recursive_interpolate
File "compose\config\interpolation.py", line 184, in convert
File "compose\config\interpolation.py", line 141, in to_int
ValueError: invalid literal for int() with base 0: '9900-9999'
Failed to execute script docker-compose
PS D:\issue> docker-compose -f ./docker-compose.yml up -d
Traceback (most recent call last):
File "docker-compose", line 6, in <module>
File "compose\cli\main.py", line 71, in main
File "compose\cli\main.py", line 121, in perform_command
File "compose\cli\command.py", line 37, in project_from_options
File "compose\cli\command.py", line 91, in get_project
File "compose\config\config.py", line 375, in load
File "compose\config\config.py", line 506, in process_config_file
File "compose\config\config.py", line 497, in interpolate_config_section
File "compose\config\interpolation.py", line 44, in interpolate_environment_variables
File "compose\config\interpolation.py", line 44, in <genexpr>
File "compose\config\interpolation.py", line 39, in process_item
File "compose\config\interpolation.py", line 39, in <genexpr>
File "compose\config\interpolation.py", line 54, in interpolate_value
File "compose\config\interpolation.py", line 77, in recursive_interpolate
File "compose\config\interpolation.py", line 74, in recursive_interpolate
File "compose\config\interpolation.py", line 74, in <genexpr>
File "compose\config\interpolation.py", line 70, in recursive_interpolate
File "compose\config\interpolation.py", line 184, in convert
File "compose\config\interpolation.py", line 141, in to_int
ValueError: invalid literal for int() with base 0: '9900-9999'
Failed to execute script docker-compose
Docker Version
PS D:\issue> docker version
Client:
Version: 17.12.0-ce
API version: 1.35
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:05:22 2017
OS/Arch: windows/amd64
Server:
Engine:
Version: 17.12.0-ce
API version: 1.35 (minimum version 1.24)
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:15:52 2017
OS/Arch: windows/amd64
Experimental: true
Docker-Compose Version
D:\issue> docker-compose version
docker-compose version 1.18.0, build 8dd22a96
docker-py version: 2.6.1
CPython version: 2.7.14
OpenSSL version: OpenSSL 1.0.2k 26 Jan 2017
Observations
- docker-compose only accepts a single port in long form syntax.
- docker currently appears to only support a single port in long form syntax. See https://github.com/moby/moby/issues/32551
PS D:\issue> docker run -p mode=host,target=9900-9999,published=9900-9999,protocol=tcp -d hello-world
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Invalid containerPort: mode=host,target=9900-9999,published=9900-9999,protocol=tcp.
See 'C:\Program Files\Docker\Docker\Resources\bin\docker.exe run --help'.
Impacts
- Customers with needs to define a large range of port bindings must declare each port individually if they need to use the long form syntax. This is burdensome and I would prefer to see parity between the long and short form syntax.
Workaround
- Customers who do not need to use the long form syntax can use the short form syntax to declare a range of ports.
- Customers who must use the long form syntax, to control which protocol is used or to bind the ports on the ingress network when deploying to swarm, need to declare each port individually in their compose files.
The error message should be improved in the next release.
As for actually supporting port ranges, the v3 format would need to be updated first - these changes happen at the docker/cli level, then get ported to docker-compose. Please open an issue there.
It also does not support not stating which port published should be used. It would be useful to be able to say published: *.
It also does not support not stating which port published should be used.
Actually, you can do that by not stating which port to publish, e.g.
version: '3.5'
services:
foo:
image: busybox
ports:
- target: 9900
protocol: udp
I found out what I did wrong: using mode: host without a published port does not show the port on portainer service overview. Did not mean to hijack the topic.
I'm running into this issue as well. Are there any updates on this?
@christopher-coffin This feature request can't be completed until the CLI supports the functionality.
If you go Thumbs Up this issue, it might get some traction? https://github.com/docker/cli/issues/839
This feature request can't be completed until the CLI supports the functionality.
I think the CLI does support it now
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}
It'd be a bit sad to see this issue closed for inactivity, so here's a comment reasserting that the CLI supports the necessary syntax, the upstream issue should be closed to clarify this.
This issue has been automatically marked as not stale anymore due to the recent activity.
This comment https://github.com/docker/cli/issues/839#issuecomment-500136821 seems to say that the problem is solved on the docker-cli side. The logical next step would be for compose to follow.
I also have same problem. When I set endpoint_mode to dnsrr, it requires ports to be declared as host mode. So I need to write some port ranges long format but it doesn't allow to do that
Confirmed not fixed in 19.03.13, on Linux at least. It still creates two iptables rules for each port in the range, instead of using a ranged rule, and more than 1000 ports takes for.........ever to complete..
Here, I exposed ports 45000-45500 using the following:
docker-compose:
...
ports:
- "45000-45500:45000-45500/udp"
...
Resulting iptables ruleset... (Should be 2 lines, not 1000)
ACCEPT udp -- anywhere anywhere udp dpt:45500
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45500
ACCEPT udp -- anywhere anywhere udp dpt:45499
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45499
...
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45006
ACCEPT udp -- anywhere anywhere udp dpt:45005
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45005
ACCEPT udp -- anywhere anywhere udp dpt:45004
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45004
ACCEPT udp -- anywhere anywhere udp dpt:45003
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45003
ACCEPT udp -- anywhere anywhere udp dpt:45002
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45002
ACCEPT udp -- anywhere anywhere udp dpt:45001
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45001
ACCEPT udp -- anywhere anywhere udp dpt:45000
ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45000
One more thing, when trying to RUN the container using port ranges as follows (per the manual examples):
docker run -it --rm -p 8002:8888/tcp -p 45000-45500:45000/udp kurento:latest
Port mapping is not done at all. The first port of each set is added only to the DOCKER ruleset, and the range forwarding is not created in DOCKER-INGRESS:
Chain DOCKER (3 references)
target prot opt source destination
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:45000
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:8888
Chain DOCKER-INGRESS (1 references)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:https
ACCEPT tcp -- anywhere anywhere state RELATED,ESTABLISHED tcp spt:https
ACCEPT tcp -- anywhere anywhere tcp dpt:http
ACCEPT tcp -- anywhere anywhere state RELATED,ESTABLISHED tcp spt:http
ACCEPT tcp -- anywhere anywhere tcp dpt:6379
ACCEPT tcp -- anywhere anywhere state RELATED,ESTABLISHED tcp spt:6379
ACCEPT tcp -- anywhere anywhere tcp dpt:8000
ACCEPT tcp -- anywhere anywhere state RELATED,ESTABLISHED tcp spt:8000
ACCEPT tcp -- anywhere anywhere tcp dpt:8005
ACCEPT tcp -- anywhere anywhere state RELATED,ESTABLISHED tcp spt:8005
ACCEPT tcp -- anywhere anywhere tcp dpt:2222
ACCEPT tcp -- anywhere anywhere state RELATED,ESTABLISHED tcp spt:2222
RETURN all -- anywhere anywhere
UPDATE: using the following command DOES add the forwarding MAPPING, but not the state or source port... so it's still busted...
docker run -it --rm -p 8002:8888/tcp -p 45000-45500:45000-45500/udp kurento:latest
Gets:
Chain DOCKER (3 references)
target prot opt source destination
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45011
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45010
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45009
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45008
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45007
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45006
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45005
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45004
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45003
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45002
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45001
ACCEPT udp -- anywhere 172.17.0.2 udp dpt:45000
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:8888
Also confirmed, fresh install of docker and docker-compose does not work (using version "3" in docker-compose.yaml)
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
This issue has been automatically marked as not stale anymore due to the recent activity.
Confirmed not fixed in 19.03.13, on Linux at least. It still creates two iptables rules for each port in the range, instead of using a ranged rule, and more than 1000 ports takes for.........ever to complete..
Here, I exposed ports 45000-45500 using the following:
docker-compose:
... ports: - "45000-45500:45000-45500/udp" ...Resulting iptables ruleset... (Should be 2 lines, not 1000)
ACCEPT udp -- anywhere anywhere udp dpt:45500 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45500 ACCEPT udp -- anywhere anywhere udp dpt:45499 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45499 ... ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45006 ACCEPT udp -- anywhere anywhere udp dpt:45005 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45005 ACCEPT udp -- anywhere anywhere udp dpt:45004 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45004 ACCEPT udp -- anywhere anywhere udp dpt:45003 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45003 ACCEPT udp -- anywhere anywhere udp dpt:45002 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45002 ACCEPT udp -- anywhere anywhere udp dpt:45001 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45001 ACCEPT udp -- anywhere anywhere udp dpt:45000 ACCEPT udp -- anywhere anywhere state RELATED,ESTABLISHED udp spt:45000
Running into this same problem.
Just commenting so Stale bot doesn't mark this as stale.
On a serious note, would love to see this resolved.
Customers who must use the long form syntax, to control which protocol is used or to bind the ports on the ingress network when deploying to swarm, need to declare each port individually in their compose files.
This is me. Specifically here I need to be able to publish a range of ports to the host in a Swarm cluster with endpoint_mode set to dnsrr (I don't want the ports to be ingress load balanced using Swarm's LB here and ingress VIPs).
Is someone working on compose support for this?
This has been (partially) fixed on the Compose-specification: https://github.com/compose-spec/compose-spec/pull/222 now need to get this also implemented in the code
This issue has been 2 years old and it is very neccessary for popular services that open range of UDP ports to comminicate like RTP protocol. Any one has a way to by pass this issue?
came here with hopium, left here with a compose file larger than life...
This issue has been 2 years old and it is very neccessary for popular services that open range of UDP ports to comminicate like RTP protocol. Any one has a way to by pass this issue?
+1
