RFC: setting up environments for services

[e0d]

For our services a subset of configuration is passed to the runtime via environment variables.

We want to easily be able to replicate the environment of the service managed by supervisor in order to run management commands and debug issues.

For this reason we have been moving away from using the environment feature of supervisor configuration in favor of a file that can be sources in the service home.

However, with the introduction of gunicorn.py files, the number of files and the dependencies between them is a little unwieldy.

service_supervisor.conf -> wrapper.sh -> service_env -> gunicorn.py

We cannot source an environment file from the supervisor.conf.

My current thinking is that adding the vars to

~/.virtualenvs/service/bin/postactivate

may be the cleanest option, though it trends a little toward the 'magical.'

Thoughts? @carsongee @jarv @feanil @fredsmith @maxrothman

[jarv]

Wouldn't this be an issue for when python is called directly in the virtualenv without using the activate script? I think in that case you would need to source ~/.virtualenvs/service/bin/postactivate which feels the same as sourcing service_env.

[e0d]

True, if python is called directly. This would only work if the virtualenv were activated. Other thoughts about making this more coherent?

[carsongee]

Can you maybe fill me in on what wrapper.sh is and why that couldn't be combined with service_env? I'm also not quite sure why there is an arrow to gunicorn.py since that is just a config file for the wsgi server and isn't really in the pipeline any more than the config for the service we are running. If so, then we also need an extra arrow to service.(json|yml|py) and one for the nginx conf file too since these are all common config dependencies for each of our services.

That said, as for ideas, we could use the service user shell profile if we ran supervisor as root and dropped to the service's user (I think?), but that has other drawbacks.

[jarv]

I think having an environment file per service is pretty clean for supervisor. For ssh'ing into servers and having a sane environment for multiple services installed on one machine I think you might take either or a combination of these approaches:

• helper scripts in /edx/bin that move you into the proper directory, set your venv and run a command
• shell functions that source the venv activate script and also source the service's environment file

Instead of helper scripts we could just have shell functions I guess if everyone was sourcing a common file though putting shell wrappers in /edx/bin was the direction we were headed previously.

[e0d]

@carsongee

Complexity around things like newrelic is handled in the wrapper script:

https://github.com/edx/configuration/blob/master/playbooks/roles/analytics-api/templates/edx/app/analytics-api/analytics-api.sh.j2

The wrapper sources the service_env script.

So, with the recent addition of the gunicorn files, which I think makes sense, we have a lot of interrelated artifacts. Trying to reduce them number to the minimum.