dbt-project-evaluator
dbt-project-evaluator copied to clipboard
Avoid `dbt run` failing if the seed has not been loaded
Describe the feature
A few people reported issues when they run the project without the seed being executed. A dbt seed
or dbt build
works (and this is what we have described in the README) but might not be super easy to catch at first.
Instead of having a "default seed" that needs to be run, I was wondering if we could remove it and use get_relation to check if the seed exists in the warehouse or not. And if it doesn't exist we just don't filter out any of the fct
tables.
Any opinion from anyone?
I am open to it! I don't think it's too much to expect folks to realize that the seed is necessary to execute, but i can also see the argument to make the package as ergonomic as possible -- seems simple enough!
What happens if you once had an exception seed (so the seed exists in the warehouse), but you then later delete the seed file from your dbt project?
I think we'd want to do something like...
{% set seed_exists = ref('dbt_project_evaluator_exceptions') is not none %}
{% if seed_exists %}
And then only apply to filter if the seed exists, but that gives me a compilation error.
Model 'model.dbt_project_evaluator.fct_staging_dependent_on_marts_or_intermediate' (models/marts/dag/fct_staging_dependent_on_marts_or_intermediate.sql) depends on a node named 'dbt_project_evaluator_exceptions' which was not found
Alternatively, we could do something like:
{% set seed_exists = load_relation(ref('dbt_project_evaluator_exceptions')) is not none %}
But we'd still need the blank seed in the package, otherwise we'd get the same compilation error.