elementary icon indicating copy to clipboard operation
elementary copied to clipboard

`edr monitor` race condition for `alerts_v2__dbt_tmp` table

Open meisam-neom opened this issue 7 months ago • 2 comments

Describe the bug Running edr monitor concurrently in production, since it is an incremental model, it does a create or replace table statement on alerts_v2__dbt_tmp. But when multiple jobs are making the same operation, we get the below error, as one job may delete this tmp table:

08:11:36  1 of 1 ERROR creating sql incremental model elementary.alerts_v2 ............... [ERROR in 26.74s]
08:11:37    Database Error in model alerts_v2 (models/alerts/alerts_v2.sql)
  Not found: Table project:elementary.alerts_v2__dbt_tmp

To Reproduce Run edr monitor in the same warehouse at the same time (timing has to be precise), so that both runs will attempt to do the create or replace statement. This way, you will see the error as one edr monitor run references the already deleted alerts_v2__dbt_tmp table

Expected behavior Since the documentation recommends running after each dbt job, there shouldn't be colisions A clear and concise description of what you expected to happen.

Environment (please complete the following information):

  • Elementary CLI (edr) version: 0.17.0
  • Elementary dbt package version: 0.18.3
  • dbt version you're using 1.9.4
  • Data warehouse: BigQuery
  • Infrastructure details: Running in prod using GitHub actions

Additional context This is probably to do with the tmp table relation config, we don't have a configuration for dbt in elementary. Ideally tmp table's should be unique for each run with the invocation id from dbt used for the naming.

meisam-neom avatar May 20 '25 20:05 meisam-neom

we have the exact same issue happening at leas once a week, same configuration as described by you

i-dalbo19 avatar Oct 02 '25 09:10 i-dalbo19

This issue is stale because it has been open for too long with no activity. If you would like the issue to remain open, please remove the stale label or leave a comment.

github-actions[bot] avatar Dec 02 '25 02:12 github-actions[bot]