`edr monitor` race condition for `alerts_v2__dbt_tmp` table
Describe the bug
Running edr monitor concurrently in production, since it is an incremental model, it does a create or replace table statement on alerts_v2__dbt_tmp. But when multiple jobs are making the same operation, we get the below error, as one job may delete this tmp table:
08:11:36 1 of 1 ERROR creating sql incremental model elementary.alerts_v2 ............... [ERROR in 26.74s]
08:11:37 Database Error in model alerts_v2 (models/alerts/alerts_v2.sql)
Not found: Table project:elementary.alerts_v2__dbt_tmp
To Reproduce
Run edr monitor in the same warehouse at the same time (timing has to be precise), so that both runs will attempt to do the create or replace statement. This way, you will see the error as one edr monitor run references the already deleted alerts_v2__dbt_tmp table
Expected behavior Since the documentation recommends running after each dbt job, there shouldn't be colisions A clear and concise description of what you expected to happen.
Environment (please complete the following information):
- Elementary CLI (edr) version: 0.17.0
- Elementary dbt package version: 0.18.3
- dbt version you're using 1.9.4
- Data warehouse: BigQuery
- Infrastructure details: Running in prod using GitHub actions
Additional context This is probably to do with the tmp table relation config, we don't have a configuration for dbt in elementary. Ideally tmp table's should be unique for each run with the invocation id from dbt used for the naming.
we have the exact same issue happening at leas once a week, same configuration as described by you
This issue is stale because it has been open for too long with no activity. If you would like the issue to remain open, please remove the stale label or leave a comment.