Downgrading migrations fails on xcom pkey

[windmark]

Apache Airflow version

Other Airflow 2 version (2.3.4)

What happened

Hi,

I'm trying to downgrade the Airflow DB migrations and the 0102_2_3_0_switch_xcom_table_to_use_run_id.py seems to not handle the xcom_pkey to pk_xcom downgrade properly.

This fails in combination with the downgrade of 0060_2_0_0_remove_id_column_from_xcom.py since the xcom table does not have an pk_xcom.

What you think should happen instead

DB migrations should be compatible with each other and downgrading 0102_2_3_0_switch_xcom_table_to_use_run_id.py should properly recreate the pk_xcom it replaces on the upgrade.

How to reproduce

1. Create an empty Airflow DB with latest migrations of anything >=2.3.0
2. Run airflow db downgrade --to-revision bbf4a7ad0465 to downgrade to 0060_2_0_0_remove_id_column_from_xcom.py

The result is the error constraint "pk_xcom" of relation "xcom" does not exist when applying the target revision. Instead it has the primary key __airflow_tmp_xcom_pkey.

Operating System

Linux

Versions of Apache Airflow Providers

apache-airflow==2.3.4
apache-airflow-providers-common-sql==1.2.0
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-sqlite==3.2.1

Deployment

Other 3rd-party Helm chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[windmark]

The issue is that the primary key name pk_xcom isn't passed explicitly as an argument and the pk thus takes the name of the temporary table __airflow_tmp_xcom_pkey. This is for some reason handled in mssql but not for example in Postgres that I'm using.

I'm however not sure how Airflow handles bugs in migrations, with backporting to already released versions. Otherwise I can help out fix the issue.

[potiuk]

Feel free to update downgrade directly in the migration that is problematic. We aren't going to backport it to anything else than 2.4.* most likely and you can simply add it to main - but check it please if it's not already fixed in main, because it could be fixed already (and not backported as we stop backporting bugfixes when we release a new minor line,).

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions[bot]]

This issue has been closed because it has not received response from the issue author.