Migration tool for Airflow config file

[Karpisek]

Description

If Im not mistaken there is currently no such tool which would provide a smooth interface for updating the airflow config to be compatible with a newer version of Airflow. Would it be feasible to create such a tool which would add new defaults to the existing config (which was compatible with prior version) and for example mark obsolete configurations, which are no longer supported?

Use case/motivation

Currently when updating to newer version of Airflow we need to go thru the config and check wether all of the configurations which we have changed are still supported in the newer versions. This is quite a tedious task.

Related issues

No response

Are you willing to submit a PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[kaxil]

We handle that from within Airflow itself. If you don't want to change the default, there is no need of adding anything to airflow.cfg.

For your use-case, do you want to know which airflow configs are deprecated in new Airflow versions? Where would you prefer this a separate section in the doc? Airflow should already log deprecations when a scheduler is started.

[potiuk]

I think it's a good idea (And marked it as good first issue). Currently we recommend (https://airflow.apache.org/docs/apache-airflow/stable/howto/set-config.html) to use this command

airflow config list --defaults > "${AIRFLOW_HOME}/airflow.cfg"

It would be quite feasible to update that command to have --update-config and rather than redirect the output, simply generate it in ~${AIRFLOW_HOME}/airflow.cfg -> which could simply leave whatever is configured and add missing defaults. If you want @Karpisek you can implement it, otherwise it will have to wait for someone to pick it up.

[potiuk]

BTW. Even now you could generate new defaults using this command and compare them / merge to make your job easier, but yeah automating it would likely be a good idea for users who prefer to see all possible values (even if their default vaiues are commented out and not used).

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions[bot]]

This issue has been closed because it has not received response from the issue author.