openstreetmap
Vagrant and Docker installations are unmaintained
Bother vagrant and docker installations suffer from constant bitrot where we add new dependencies or change configuration files but don't update the relevant setup scripts.
Unfortunately we have no way of detecting this until some unfortunate end user tries to use them because none of the main developers uses them.
We either need to find a way of testing them (is that even possible in travis? or is there some other way?) or we need to admit defeat and remove them.
So vagrant can't be done in travis apparently (https://github.com/travis-ci/travis-ci/issues/6060) so we can forget that at least.
I'd prefer to develop in a docker environment. I can work on creating a new Dockerfile. Do you have any requirements? My plan is to create 2 images, one for db and one for rails. For me this may have to wait until after SotM US.
@openbrian : see https://github.com/openstreetmap/openstreetmap-website/pull/2272
Yes this ticket is probably a bit confusing because I opened it thinking we already had a docker config but apparently my memory was faulty... Though it's definitely true that it would be good to have it tested to stop it breaking. That's something that I think travis can do, unlike vagrant.
So I don't think we really need the vagrant/virtualbox bit tested, so much as the rest of the setup script i.e the contents of /script/vagrant/setup/provision.sh. Could that be tested on travis?
Of course, both the travis config and the vagrant setup (and the Docker setup) cover a lot of the same ground - installing postgres, bundle install, etc. So if there's a way to refactor things to share some code, that would be great too.
just find this page where is seems docker builds can be tested with travis. Though needs to take a closer look at this.
I'd prefer to develop in a docker environment. I can work on creating a new Dockerfile. Do you have any requirements? My plan is to create 2 images, one for db and one for rails. For me this may have to wait until after SotM US.
@openbrian please give your feedback on #2272 as I am just a novice on docker. :blush:
For reference, just adding a note here that I'm attempting to address the Docker part of this in PR https://github.com/openstreetmap/openstreetmap-website/pull/2409
Travis is no more, so the above is a bit outdated. Time for a reassessment? 🙂
I see that a GitHub action was added (.github/workflows/docker.yml) to test the Docker image, so perhaps that's this problem solved?
Having said that: bear in mind that the mentioned action is imperfect as it runs ubuntu-latest AMD64, and not other host systems. Normally this wouldn't be a problem, but currently the tests fail on Docker with a macOS ARM host (see https://github.com/openstreetmap/openstreetmap-website/issues/6287, PR incoming). From what I can see, it's not possible to run this combo (Docker+macos+ARM) on GitHub Actions: https://github.com/orgs/community/discussions/25777.
As for Vagrant, my impression is that it's possible to run it on GitHub Actions, but requires "large" runners, which are paid.
@gravitystorm - Re: testing script/vagrant/setup/provision.sh: unless I misunderstand, I think that's not feasible. The script installs apt packages and copies config files that are different from the Docker ones. The environments in Vagrant and Docker are different, so testing the script in GitHub's Docker environment won't ensure that it works, and in fact could fail when it's correct.
Any idea of how common it is for contributors to use Docker or Vagrant? My impression is that Docker is useful and Vagrant is out of fashion... but that's my 100% anecdotal experience so I'd love to hear opinions.
I'm not sure vagrant has ever got much use and I'm pretty sure that these days if people try to use something then it's docker.
I don't really like the docker setup myself because it doesn't seem to me like a good fit for development where you want to be able to change things - it's especially weird that we bake npm modules into the docker image so that they can't easily be changed but leave gems in the host volume where they can be. At least I think that's the current state. Personally I'd also prefer it if we only had a single container so we didn't need compose.
I suspect the answer is to ditch vagrant and improve docker and then add devcontainer support using it as suggested in #5264.
I suspect the answer is to ditch vagrant and improve docker and then add devcontainer support using it as suggested in #5264.
My strategy was going to be to set up devcontainers and then ditch the other two 😄
Well I thought devcontainer was something on top of docker which is why I suggested what I did.
I had a play with dev containers this afternoon (https://github.com/openstreetmap/openstreetmap-website/compare/master...pablobm:devcontainer). It appears to work and it can reuse the existing Dockerfile and docker-compose, which would spare us having to write new provisioning scripts (packages, bundler and yarn, etc), plus we need another container for the DB. Then I realised that @mmd-osm did the same thing last year: https://github.com/openstreetmap/openstreetmap-website/issues/5264, and that Rails appears to offer a generator for a more "Rails official" way of doing this. I'll have to look a bit more into it, including how to integrate it with GitHub Actions.
https://github.com/openstreetmap/openstreetmap-website/pull/6424#issuecomment-3409931045 (emphasis added at end is mine)
I'm going to change tack slightly and not remove the "manual" Docker config with this PR yet. My concern is about how widespread devcontainers actually are, as they require certain tool support and the manual methods to work with them are not trivial. I want to make sure we don't remove something that is working and useful and replace it with something less well understood that might be trickier to use.
To clarify: I don't mean that devcontainers are definitely more complex, just that I don't understand whether they are or not.
I would like to hear opinions before removing the old way of using Docker.
Thank you @danieldegroot2 for thinking of pinging here 🙂
To fully phrase the question: team, who uses the Docker setup? What is your experience with it? Do you have any experience with devcontainers?
Since we are at it, it would be great if people try out https://github.com/openstreetmap/openstreetmap-website/pull/6424 😇 It provides support for devcontainers.