dbt-databricks icon indicating copy to clipboard operation
dbt-databricks copied to clipboard

Materialized view refresh in progress is returned as an error

Open kdazzle opened this issue 1 year ago • 1 comments
trafficstars

Describe the bug

A clear and concise description of what the bug is. What command did you run? What happened?

When the model for a materialized view is run, but that same model is already currently running in the background, the job returns as a failure. I would expect this to be a warning, since there was no actual error.

dbt run --select="tag:hourly-jobs" --target=prod

Steps To Reproduce

Run a long-running materialized view model that already exists (not a full-refresh). And while that is still running, run it again.

Expected behavior

I would expect it would return as a warning instead of a runtime error, since it's not entirely unexpected - it's just running a long-running job.

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

Runtime Error in model latest_vehicle_location (models/marts/my_mart/my_materialized_view.sql)
  A refresh is already running in the background. You can monitor the previous refresh by visiting the following URL:

System information

The output of dbt --version:

23:00:28  Running with dbt=1.7.8
23:00:30  Registered adapter: databricks=1.7.8

The operating system you're using: dbt cloud

The output of python --version: dbt cloud

Additional context

Not a huge deal, but it creates noise that could hide real errors

kdazzle avatar Mar 01 '24 23:03 kdazzle

Can you validate if this error is still present in 1.8.0b1? I suspect that it is, but since that release has many changes to how MVs work, would really appreciate the extra info.

benc-db avatar Mar 04 '24 17:03 benc-db