compose
compose copied to clipboard
docker
allow removing something in docker-compose.override.yml
My usecase is that the base docker-compose.yml maps a certain port in the ports section. In my docker-compose.override.yml I'd like to change that port since I already have a different service running there on my host.
As far as I understand the current implementation it is only possible to add stuff, override extends the other file.
As far as I can tell there is no way to remove an entry. And I can't think of an obvious syntax for it.
Correct. Best practice is to not include anything in docker-compose.yml that you might not want to have in another environment.
My problem with your suggestion is that docker-compose.override.yml might not exist (it's an optional file the developer can create locally). I only want to use docker-compose.override.yml to locally customize ports, like this:
docker-compose.yml:
version: '2'
services:
web:
ports:
- 80
- 443
...
docker-compose.override.yml:
version: '2'
services:
web:
ports:
- 32080:80
- 32443:443
This isn't a big deal because this will only result in 80 being exposed twice, once as a random port and once as 32080. I think it would make sense in this case to only expose it as 32080 automatically (without having to specify anything in the override to remove the original config value)
Facing the same problem. I've got one specific host which shouldn't start the one of the docker containers for given service. Yeah, maybe my docker-composer.yml approach is not the best, but I can see the need of removing something.
Alright, there is no solution so far, so I got a simple port-remover https://github.com/paunin/docker-compose-v2-ports-remover
In general I dont think docker-compose should allow to do it, as then logic of merging many files is gonna be complex...
This was already solved here https://github.com/docker/compose/pull/3939
Having an "override" thing that cannot do half of the job... well, it's something.
Same issue, I'd like to have possibility to remove port, dns, etc in docker-compose.override.yml but not override or extend with another one..
Anyone notice that the only thing that you can't do with docker-compose.overide.yml, is to actually override anything!
I would like to remove some services because I rarely use them on my machine, so I ended up with the following docker-compose.override.yml:
version: '3.5'
services:
elasticsearch:
image: hello-world
selenium:
image: hello-world
To add two more cents to the pile. It can happen that you pull a docker-compose.yml from another group and want to not run some of what they have turned on. Please add some syntax for removing.
Stumbled upon this issue when trying to override the dns settings of a random Github project without modifying their docker-compose.yml. The new dns servers are appended to the original list instead of overriding the original ones - as I supposed it will work based on the file name. This behavior is misleading IMHO.
Same problem here... It seems it's not overriding some values correctly. Can't remove either.
For example i'm overriding with network_mode: host and need to remove links, because otherwise it's throwing error:
ERROR: for processor Cannot create container for service processor: conflicting options: host type networking can't be used with links. This would result in undefined behavior
Setting links: [] doesn't help, links: null or links: ~ throws format error.
If the above saying is true, it means that the inherent parser is not able to just load several dictionaries and then apply the overriding of keys following some arbitrary order, which is not complicated in nature.
The truth is that the current design has made a key dependent behaviour. ref https://docs.docker.com/compose/extends/
My feeling is that it would be better to go for a more intuitive behaviour for developers, like that in multiple inheritance cases. The current use case consists of combining multiple yaml files in the same command line, which looks just like a one layer inheritance. The better would even be able to support multiple layer multi-inheritance using some kind of inclusion key word. Also, to specify the case when child will either merge or simply replace, some key words like "super" could hence be interesting to have.
"Composition" could be a workaround to achieve the same thing as "inheritance". In the case of docker compose file one may do something similar. The key word "extend" becomes helpful in this case, which may , yet, result in a more verbose compose file. Also, one would need to very carefully craft each file so that things are always extended/override but not removed.
#!/usr/bin/env ruby
require 'yaml'
data = YAML.load_file "docker-compose.yml"
majorVersion = data['version'][0].to_i
serviceName = ARGV[0]
image = ARGV[1]
if majorVersion == 3
data['services'][serviceName]['image'] = image
data['services'][serviceName].delete('build')
else
puts 'Version #{majorVersion} not supported yet. Please ask your cloud-ops team!'
exit 1
end
File.open('docker-compose.yml', 'w') { |f| YAML.dump(data, f) }
Needed a small script which removes build and adds image to a docker-compose service. Maybe this helps other building CI scripts as well e.g. removing a service.
From my point of view the remove feature should be added to docker-compose overwrite as it decreases user experience of docker-compose a lot.
Hi Thomas,
I wrote a simple script, docker-compose-use, as the basis of my AngelBox toolkit. This makes docker-compose files "composable". You will find this at http://github.com/keithy/sprankle-pod and http://github.com/keithy/angelbox
@thomaspeitz or just use yq. you can easily edit parts of yaml with it.
F.e. Adding image: yq w -i ./docker-compose.yml services.nginx.image nginx:latest
or removing ports: yq d -i ./docker-compose.yml services.db.ports
It is ridiculous that the team of "professionals" of docker-compose cannot solve this easy issue for years. And even more funny that override file cannot even properly override anything.
I also vote for this. Already tried adding override_strategy: overwrite in the given yaml-section which won't work (anymore?!) mentioned here #2260
I would also vote for such an option which also would resolve deleting by setting a property to "[]" or "" to empty it.
I'm still hoping this gets addressed. The best solution for me is using "image: hello-word" for unneeded services but it's so smelly. The only alternative is to ditch the override feature completely and duplicate much of the original docker-compose.yaml into a completely alternate version for production...not DRY or ideal for environment consistency but seems to be the standard practice.
The best solution for me is using "image: hello-word" for unneeded services but it's so smelly
Option that might be slightly less smelly? - we tend to leave the image, etc as-is and do something like entrypoint: /bin/sh in the override file so the container runs sh then exits (rather than running its usual entrypoint/cmd).
It is ridiculous that the team of "professionals" of
docker-composecannot solve this easy issue for years. And even more funny that override file cannot even properly override anything.
Truly remarkable and only one of the many. Why not substitute it with a map instead? It's not like you can bind to the same host port twice. Why not a map where you could easily say, override it to "nothing" for example by doing 8080: -
ports:
8080: 80
21000: 5432
# ... in some other file extending the prev config
ports:
8080: -
No wonder why people move so passionately to Kube.
I just stumbeld accross this issue while searching a way to override a port. Kinda sad that this isn't possible. Semms to me quite essential.
Because this isn't yet implemented, I had to create to copies of docker-compose.yml, remember to use the correct one at any given time, and now update both whenever I want to make a change that is irrelevant to environment.
My use-case is elasticsearch and TLS. Dev uses a self-signed cert so needs a root CA and SSL verification mode set to certificate only, whilst Prod uses letsencrypt TLS set up during provisioning.
Here are my dev environment vars:
- xpack.security.enabled=true
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/key.pem
- xpack.security.transport.ssl.certificate=certs/cert.pem
- xpack.security.transport.ssl.certificate_authorities=certs/rootCA.pem
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/key.pem
- xpack.security.http.ssl.certificate=certs/cert.pem
- xpack.security.http.ssl.certificate_authorities=certs/rootCA.pem
- xpack.security.http.ssl.verification_mode=certificate
Here are my prod vars:
- xpack.security.enabled=true
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.key=certs/key.pem
- xpack.security.transport.ssl.certificate=certs/cert.pem
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=certs/key.pem
- xpack.security.http.ssl.certificate=certs/cert.pem
If I try and run prod with any of the ssl.certificate_authorities vars, even as empty strings, elasticsearch will not start. They should not be empty strings, they simply shouldn't exist.
Given this use-case, I now have to have a dev docker-compose and a prod docker-compose, which are completely identical bar the above change in environment variables.
It would be very useful to not have to do something like this.
My use case for this missing feature is to have development and production setups.
By default, I want to just type
docker-compose up
and be in development mode. This bind/mounts the source code over the application within the container so I can edit and see the changes happen within the container immediately.
When I go into production mode, I (think I) have to do something like this:
docker-compose -f docker-compose.yml -f docker-compose.override.yml -f docker-compose.production.yml up
And the production.yml would remove the binds.
There are probably better ways to achieve this, but this is a real world use case.
I mean, I could have a .development.yml file that I always have to use in development and if I don't it's production. But then I'm typing the -f x3 all the time, when the short docker-compose up is what I'd like to type 99% of the time.
You need to have a look at my docker-compose-use script see github.com/keithy/sprankle-pod a more advanced form is available in https://github.com/keithy/angelbox