airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Remove unused index `idx_last_scheduling_decision` on `dag_run` table

Open pankajkoti opened this issue 1 year ago • 0 comments

We added idx_last_scheduling_decision on the last_scheduling_decision column in the dag_run table with Airflow 2.0.0. However, this index seems to have been unused with 0 index_scan counts throughout. I verified scheduler performance metrics are not affected after removing this index.

Additionally, this index seems to occupy huge storage space, almost half the size of the table (e.g. on one deployment that I checked, the table occupies 9.8GB of records, but this single index alone occupies 5.7 GB)


^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

pankajkoti avatar Apr 26 '24 09:04 pankajkoti