apache
Snowflake SQL API hook Retry Logic
Updated sync and async Snowflake API call methods to retry on errors as advised by Snowflake (https://docs.snowflake.com/en/developer-guide/sql-api/handling-errors) and on connection related errors.
closes: #50514
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.
@mik-laj or @potiuk, would it be possible to get a review of this soon? I'm working with a few teams that heavily use the Snowflake SQL API and this would help their DAGs run much more smoothly!
Hey @mik-laj, would you be able to take another look at this soon? Thank you!
Hey @mik-laj, I've updated the implementation to use context managers for both the http session and the retrying object. Thank you for the feedback and let me know what you think of the updates 🙌
Hi @astro-anand do you happen to know when this will be released?
We usually release provider's every 2 weeks (See the README). We have one release in progress (see devlist) so you might expect the next wave within 2 weeks.
But you can also - and we encourage you to - to build your own provider, it's super easy (there are a number of ways - starting from breeze and ending with just running flit build in the providers folder (this is just absolutely standard python package with it's pyproject.toml`) - and installing it locally for tests.
Also when voting on the new provider starts (you should observe devlist and there will be github issue) you are encouraged to test and see the RC candidate (and let us know when you did)
