concourse
RFC: Services
My preference would be to maintain the standard docker-compose configuration. Perhaps the service key can point directly to a docker-compose file. That way one could easily use a docker-compose file in a git repo if needed. Concourse would also render any variables in this compose file like it does with task files.
I guess the issue with this approach would be networking configuration may not match up with what concourse can supply. I would propose that concourse creates a compose container such that the docker host network is internal to the compose container.
@samgurtman by "maintain the standard docker-compose configuration", do you mean be able to use your existing docker-compose.yml files as services? If so, there's nothing preventing the use of running docker-compose in a container as we currently do - but using services means that the task you run alongside it can be unprivileged.
In the RFC (the 2nd snippet under Service Configuration), I gave an example of running a service using a theoretical docker-compose Prototype, but you could also use one of the existing "docker-compose in docker" images as a service instead. This proposal just doesn't force you to use docker-compose for your services, but it's certainly possible to do so.
I guess the issue with this approach would be networking configuration may not match up with what concourse can supply
I think running docker-compose in a container should be fine, as long as your docker-compose.yml exposes ports, as Concourse won't be able to docker host network (as you mention)
As the image in https://github.com/aoldershaw/rfcs/blob/services/084-services/proposal.md#networking states port 8080 specifically, I just wanted to point out that there are images
- exposing a different port
- exposing multiple ports
MySQL is a good example for that.
@elgohr fair point, it's confusing - an earlier form of the diagram was tracing a single request to a specific example service endpoint, and I never ended up changing the port from 8080. Updated the diagram.
Worth noting that the use-case of different/multiple ports is supported - you can configure as many ports as you wish when configuring your service (https://github.com/aoldershaw/rfcs/blob/services/084-services/proposal.md#service-configuration). e.g.
task: ...
services:
- name: mysql
...
ports:
- name: default
number: 3306
- name: admin
number: 33062
I did an initial implementation of the Services RFC. https://github.com/concourse/concourse/pull/8673. I'd want some feedback from Concourse maintainers before finishing it, so I don't waste my time if they won't accept it. It works great so far.
the only thing I'd like to see, in addition to launching per task services, would be allowing tunneled access to services per worker, that can be assumed to exist.
I.e. a sccache-server/redis instance for caching compile artifacts across builds and tasks.
It would fit the tag on the can, but I am not convinced it's a) a good fit for the concept or b) bloating the RFC too much and should be extracted/not included.
