docker-airflow icon indicating copy to clipboard operation
docker-airflow copied to clipboard

How to keep Airflow Variables when restarting the docker image?

Open 94929 opened this issue 4 years ago • 2 comments

In Airflow UI, I created airflow variables via Admin > Variables > Create. However, every time I restart the docker image, the airflow variables set are disappeared. So I need to re-create all the variables I created, which is somewhat annoying.

Is there a way to keep those variables (make it non-volatile) so that I can use them even in restarting the docker image?

94929 avatar May 29 '21 05:05 94929

Have you tried committing the changes to the docker image and then referencing the committed image in the yaml file? Commit changes to a docker image: https://docs.docker.com/engine/reference/commandline/commit/ Change yaml file to point to new image - this should be referencing the newly committed image name image

foxviii avatar Jun 05 '21 20:06 foxviii

@jha929 @foxviii hi there, hopefully still in time I dealt with this issue creating a makefile which loads vars copied during build time OR added via docker cp / volumes. Vars could be loaded running make vars

  • Dockerfile: COPY variables.json /tmp/variables.json

  • Makefile:

vars:
	docker compose exec webserver airflow variables -i /tmp/variables.json 

metaver5o avatar Jul 26 '21 12:07 metaver5o