Cannot reference airflow_local_settings in config settings

[rauljmz]

Official Helm Chart version

1.6.0 (latest released)

Apache Airflow version

2.3.3

Kubernetes Version

1.24.0

Helm Chart configuration

config:
  database:
    sql_alchemy_connect_args: airflow_local_settings.keepalive_kwargs

airflowLocalSettings: |-
  keepalive_kwargs = {
    "keepalives": 1,
    "keepalives_idle": 30,
    "keepalives_interval": 5,
    "keepalives_count": 5,
  }

Docker Image customisations

No

What happened

I am trying to pass some different keepalive args to the SqlAlchemy connection as per the documentation recommendation. (See the values.yaml snippet added to the issue). With this change in place the chart fails to upgrade as the database migrations never complete.

What you think should happen instead

Airflow should complete database migrations and start up normally

How to reproduce

Just do a helm upgrade --install with the values.yaml snippet above.

Anything else

The problem occurs because the airflow_local_settings.py is not mounted to the migrations job. This however can be hacked by adding this snippet to the values.yaml:

migrateDatabaseJob:
  extraVolumeMounts:
    - name: config
      mountPath: /opt/airflow/config/airflow_local_settings.py
      subPath: airflow_local_settings.py
      readOnly: true

But then the chart still refuses to start, as the init containers waiting for the migrations in the webserver, triggerer and scheduler deployments, also miss the same airflow_local_settings.py file. Since they use the airflow CLI to check the db for migrations they fail to execute.

Are you willing to submit PR?

• [X] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!