airflow
airflow copied to clipboard
Add Iceberg provider
Following https://github.com/apache/airflow/pull/32786#issuecomment-1768284013 since Tabular provider doesn't have any specific tabular logic it has been suggested to move Tabular logic to new Apache Iceberg provider
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst
or {issue_number}.significant.rst
, in newsfragments.
@Fokko Do we want to keep Tabular provider (if there might be future tabular specific behavior?) or should we deprecate the whole tabular provider in favor of apache-iceberg provider? (difference between deprecate specific classes vs the whole provider)
@Fokko Do we want to keep Tabular provider (if there might be future tabular specific behavior?) or should we deprecate the whole tabular provider in favor of apache-iceberg provider? (difference between deprecate specific classes vs the whole provider)
No, let's deprecate it in favor of the Iceberg provider. Everything in here is just Iceberg, and nothing Tabular specific, so the current naming is confusion
@romsharon98 Sorry for being late to the party here, but this looks great! Thanks for picking this up 🎉
@eladkal That makes sense. It was not my intent to merge this without your approval, but just to indicate that there are no further comments on the Operator and Hook itself.
@eladkal That makes sense. It was not my intent to merge this without your approval
I know :) This is just to avoid merge by another committer. We are 99% ready here. it's just for the procedure of release due to https://github.com/apache/airflow/pull/39240 for simplicity I will handle it as ad-hoc release
🎉🎉🎉
Wooohooo!
Nice! Thanks @romsharon98 for working on this, and thanks @eladkal and @potiuk for the review 🎉