docker-airflow icon indicating copy to clipboard operation
docker-airflow copied to clipboard

Local Executor configuration issues

Open Schachte opened this issue 4 years ago • 2 comments

I spent several hours trying to get the local execute docker-compose file running on my external server and finally have it working. I noticed I needed to do a lot of things that weren't configurable via the compose file out of the box..

Interested to hear thoughts on this:

  • I needed to rebuild the Dockerfile with the postgres connection string for Sql Alchemy
  • I needed to manually change the executor type to LocalExecutor
  • I needed to build a shared network in the compose file so the airflow container could communicate to the PostgresDB
  • In addition to this, I needed to ensure that the hostname for the DB connection string used the service DNS (ie postgres instead of localhost)

This required a bit of debugging, but wondering if we should update the airflow.cfg to take in env params for the executor type and db connection strings and also add a shared network interface between all the services to enable communication.

I'm happy to raise a PR if this seems reasonable. Thanks.

Schachte avatar Apr 19 '20 19:04 Schachte

The entrypoint script is responsible for these adjustments.

https://github.com/puckel/docker-airflow/blob/master/script/entrypoint.sh

wittfabian avatar Apr 20 '20 06:04 wittfabian

I have the same problem, the EXECUTOR=Local does absolutely nothing to the config file, my current workaround is to mount the config file as volume, please address this issue soon.

vlizanae avatar Sep 10 '20 23:09 vlizanae