A better Decidim in Docker

[froger]

Is your feature request related to a problem? Please describe.

An official docker support and documentation is needed to run easily a Decidim Instance. In the current state, my concerns about docker are:

• It needs to be ready to be run with sidekiq, puma and others (activestorage/activecable).
• It needs more env variables to be configurable (caching/files in s3)
• It needs some docker-compose to show integrations with initiative time-stamping, etherpad and e-voting
• It needs some minor security improvements, to match common docker configurations for production

Describe the solution you'd like

Remove decidim/docker repository that uses the generator

• Improve decidim-generator
• A Dockerfile production-ready
• Add binaries to run: sidekiq/puma/actioncable/actionstorage.
• Add env variables to configure the container: caching, files in S3, logs rotation, etc.

Additional context See Metadecidim proposal

Acceptance criteria

• [ ] Given that I'm a developer, When I generate a new decidim instance Then I can run a docker instance securely in a one line script.
• [ ] Given that I'm a developer, When I run a fresh decidim instance Then I can configure cache, file uploads, logging and async jobs.
• [ ] Given that I'm a developer, When I run docker scan Then I see no security issues or warning
• [ ] Given that I'm a developer, When I see the docker-compose.yml Then I see all the necessary dependancies to run a full-featured decidim.

[froger]

To give some contexts on how the work is going to be proposed, I plan to split the work on the following PR:

1. Pull Request to propose changes in the generator in order to configure the app with Environment Variables and Secrets. (with documentation).
2. Pull Request to propose Dockerfile for production, with documentation on how to run it
3. Pull Request to propose docker-compose files:
   • a slim version: what is minimal to run an instance
   • a full-featured version: what can you do to run a fast and secure installation.

My strategies are :

• Secure by default
• Stay as small as possible
• Follow a classical naming conventions based on references

My references for docker implementations:

• Ory/Hydra: OpenID Certified™ OpenID Connect and OAuth Provider written in Go
• Solidus: A free, open-source ecommerce platform written in Rails
• Metabase-compose: Docker configurations for metabase

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. @carolromero & @andreslucena feel free to chime in.

[oxcar]

Is somebody working on this?

[andreslucena]

Hi @oxcar!

At the moment there isn't anyone working actively on this one as far as I know. This would be an awesome contribution to the project, for sure. A good starting point would be where @froger arrived: https://github.com/octree-gva/decidim-in-docker

[andreslucena]

@froger after our meeting the past week, I understand that you guys have been working on this issue. Do you have any update?

[froger]

Hello @andreslucena , we have make some good progresses. Let me describe here our adventure.

Hard but solved stuff

Working defaults

Decidim in docker needs to takes some decisions about job processes supervisions, caching, async jobs etc to have working defaults. decidim_generators is quiet generics, so we need to:

• Generate an app with decidim_generators
• Apply working default
• Be enough flexible to don't require work on each release.
• Be able to go from a version to another

So what we do is: use decidim-generators inside a docker image the, add some working defaults, wrap all this together and expose a minimal image (but still big, because deps are big).

Volumes

The hosters we use (jelastic/digital ocean) will empty volumes on mounting, and we often have issues on empty /public directory. Still somehow an issue, but this isn't a blocking issue.

Default user

After some security reviews from Geneva, we really tried hard to have "not" the default user running the container. This was a bit tricky, but we did it.

Opinionated Decisions

Docker image use supervisord by default

bundle exec rails s is not really a reliable solution to run rails in docker, as it won't restart if it fails. We started to use systemd as sidekiq wiki suggest, but we found the configurations over-complicated. So we've switched to supervisord and it looks better. This help us to be able to run multiple processes in the same image: puma and sidekiq will be the default.

# run sidekiq and rails
docker run <dockerhub>/decidim -v ./logs:/home/decidim/app/logs -e RUN_SIDEKIQ=1 -e RUN_PUMA=1
# run just rails
docker run <dockerhub>/decidim -v ./logs:/home/decidim/app/logs -e RUN_SIDEKIQ=1 -e RUN_PUMA=0

Bundle some docker-compose

Decidim without at least postgres and redis offers a very poor experience. We are setting up some docker-compose to have a getting started experience similar to Ory/hydra. Things like:

# Let's run a decidim!
git clone <github-repo>/decidim/docker
docker-compose -f quickstart.yml -f ehterpad.yml up -d
# Visit localhost:3000 to see an empty organisations

Docker image don't serve assets

Docker image should be performance-ok by default, we won't serve assets, and provides a Nginx configuration by default. (Open Source Politics have published working Nginx config somewhere). So the quickstart.yml docker-compose includes some Nginx instance by default. (aside: this also help us to demonstrate working with private networks to don't expose dbs and processes).

Docker image requires running postgres and redis to start

Async tasks with sidekiq and caching will use redis instance (tried to run decidim without cache, that's not a nice user experience). At the docker image's entrypoint we requires a running postgres host and redis host through a wait-for-it scripts.

Why the heck you didn't publish anything?

My cute excuse: Docker for decidim will improve the getting started for all future decidim users. We need to get it done well and we need to take times because I'm sensing this is an important job to be done.

Truely:

• We work exclusively on 0.24 versions, and I need to extensively tests all the features to avoid surprises (wkhtmltopdf install nightmare for example).
• I've also been very unsatisfied on docker performance for rails, so I've tested some alternative ruby runtimes (Truffle Ruby and Fullstack ruby). This side job was time consuming, and as I don't think Decidim should propose docker image based on alternative ruby runtimes, this wasn't really good for the issue's purpose to explore this.
• I use Gitlab and have absolutely no ideas on how Github workflows works. (help?)

All of this to say sorry didn't propose a working PR for the docker installation.

That's all :) Our timeline is now December with develop branch sync, but as we have something working for 0.24 I will publish a PR ASAP.

See you at the fest!

[froger]

To follow status, here a PR in draft: https://github.com/decidim/docker/pull/98