telemetry-airflow
telemetry-airflow copied to clipboard
mozilla
Add Glam DEV legacy telemetry dag to start testing hot probes list
This creates a new Dev DAG for Glam, which will be used to test the modified bigquery-etl code (not yet landed) that only computes probes that have been recently created or searched for on Glam.
Tested locally and confirmed the new dag gets created.
@edugfilho have you done a test run and confirmed that this does not interfere with production ETL (i.e. does not write to the same tables)?
Good point. This is something to be done in the bigquery-etl project. I'm supposed to do this today.
@edugfilho have you done a test run and confirmed that this does not interfere with production ETL (i.e. does not write to the same tables)?
Good point. This is something to be done in the bigquery-etl project. I'm supposed to do this today.
Tested clients_daily tables manually. Others are being tested but are less risky because they read and write to the same dataset. All tasks under glam_dev have the dataset_id set to dev_telemetry_derived