bryzgaloff
Change requirements airflow max version ~2.4
@was-av could you please provide a fix for a failed test? Unit test test_get_hook_defined fails: AttributeError: module 'airflow.sensors' has no attribute 'sql'.
In general, we should create a dependency matrix with the following relationship:
- Python version.
- Airflow max version.
- ClickHouse plugin max version.
And test them accordingly. In this case users of earlier Python/Airflow versions will be able to pick up a right version of the plugin.
@was-av will you be able to do this as a part of a separate PR? Our workflow config currently is too generic matching improper combinations of the versions.
Could you provide more details on how to implement such a matrix?
Simply a markdown table. This requires a manual check of different Python/Airflow/plugin versions compatibility. Something like:
| Plugin version | Max Airflow version | Max Python version |
|---|---|---|
| 0.8.2 | 2.3.* | 3.10.* |
| … | … | … |
And the .github/workflows/tests.yml versions matrix should list the same combinations, so that we know the tests correspond to a list of versions.
BTW, are any Python or Airflow versions not supported already completely? We do not need to support Python versions earlier than Airflow supports.
In general, this should be a PR prior to yours. So that we first have to specify this matrix and then easily integrate your version increment PR into this. Potentially I may compose such a matrix this week, so will let you know if done.
Hi @SergeyBychkov we are currently stuck with absence of versions matrix, could you please make another PR with it? https://github.com/bryzgaloff/airflow-clickhouse-plugin/pull/44#issuecomment-1296302184
Just a research over the documentation needed for the most recent Python versions (which are still officially supported), Airflow versions, and, potentially clickhouse-driver versions. For every combination the latest possible airflow-clickhouse-plugin version should be matched. Then we are able to introduce support for the latest version of Airflow easily.
Guys, good news: Airflow 2.4 support has been added as a part of #50
@was-av I am going to close this PR. But I will mention you as a contributor on README anyway, thank you for your participation!
Airflow 2.4 support is released: https://pypi.org/project/airflow-clickhouse-plugin/0.9.0/
Though, Airflow 2.1 support is disabled, details: #52