telemetry-airflow icon indicating copy to clipboard operation
telemetry-airflow copied to clipboard

Published 20 hours ago • verified publisher icon mozilla

Update Fivetran connector to v1.1.2

Open alekhyamoz opened this issue 2 years ago • 2 comments

The PR would resolve the issue where the syncs are completed quickly after being started by the FivetranOperator. More details here: https://github.com/fivetran/airflow-provider-fivetran/issues/60

The Fivetran connector v1.1.2 will help us avoid the task priorities set in private_bqetl_revenue DAG: https://github.com/mozilla/private-bigquery-etl/blob/main/dags/private_bqetl_revenue.py#L152-L154

There would be a subsequent PR to make the necessary changes to the revenue DAG

alekhyamoz avatar Jul 25 '22 14:07 alekhyamoz

Can you create a DSRE Jira for installing the fivetran airflow provider 1.1.2 on the wtmo cluster? I can add this to a list of needs for when I upgrade airflow again.

haroldwoo avatar Jul 25 '22 18:07 haroldwoo

@haroldwoo Created a Jira ticket :https://mozilla-hub.atlassian.net/browse/DSRE-921

alekhyamoz avatar Jul 25 '22 18:07 alekhyamoz