docker-airflow
docker-airflow copied to clipboard
Local Executor configuration issues
I spent several hours trying to get the local execute docker-compose file running on my external server and finally have it working. I noticed I needed to do a lot of things that weren't configurable via the compose file out of the box..
Interested to hear thoughts on this:
- I needed to rebuild the Dockerfile with the
postgres
connection string for Sql Alchemy - I needed to manually change the executor type to
LocalExecutor
- I needed to build a shared network in the compose file so the airflow container could communicate to the PostgresDB
- In addition to this, I needed to ensure that the hostname for the DB connection string used the service DNS (ie
postgres
instead oflocalhost
)
This required a bit of debugging, but wondering if we should update the airflow.cfg to take in env params for the executor type and db connection strings and also add a shared network interface between all the services to enable communication.
I'm happy to raise a PR if this seems reasonable. Thanks.
The entrypoint script is responsible for these adjustments.
https://github.com/puckel/docker-airflow/blob/master/script/entrypoint.sh
I have the same problem, the EXECUTOR=Local
does absolutely nothing to the config file, my current workaround is to mount the config file as volume, please address this issue soon.