elementary icon indicating copy to clipboard operation
elementary copied to clipboard

Error when trying to run report for source freshness

Open jpnalvim opened this issue 1 year ago • 3 comments

When I run edr report, I have a test in my source set up like this:

sources.yml:

version: 2

sources:
  - name: mysource

    schema: mydataset
    tables: 
      - name: table1
        loaded_at_field: "CAST(consumer_etl_date_time AS TIMESTAMP)"
        freshness:
          warn_after: {count: 3, period: hour}
          error_after: {count: 3, period: day}      
        meta:
          elementary:
            timestamp_column: "CAST(consumer_etl_date_time AS TIMESTAMP)"
        tests:
          - elementary.freshness_anomalies
          - elementary.all_columns_anomalies:
              column_anomalies:
                - null_count
                - missing_count
                - zero_count

I get the error in the report:

elementary_source_freshness_anomalies_ingestion_layer_table1_ (models/ingestion_layer/sources.yml) dbt was unable to infer all dependencies for the model "elementary_source_freshness_anomalies_ingestion_layer_table1_". This typically happens when ref() is placed within a conditional block. To fix this, add the following hint to the top of the model "elementary_source_freshness_anomalies_mysource_table1_": -- depends_on: {{ ref('monitors_runs') }} > in macro test_freshness_anomalies (macros/edr/tests/test_freshness_anomalies.sql) > called by test elementary_source_freshness_anomalies_mysource_table1_ (models/mysource/sources.yml)

Where should I exactly add -- depends_on: {{ ref('monitors_runs') }}

Is there a way to not have to do that? The documentation does not mention that.

Also

ALL INVITE LINKS TO SLACK ARE OUT! I can't get in

jpnalvim avatar Nov 08 '23 21:11 jpnalvim

@erikzaadi can you or someone help out here? looks like all of the depends_on hints are added by default on the elementary macros. What else should I do on my side? This happens with a lot of other tests.

  To fix this, add the following hint to the top of the model "elementary_schema_changes_table1_":
  
  -- depends_on: {{ ref('schema_columns_snapshot') }}
  
  > in macro test_schema_changes (macros/edr/tests/test_schema_changes.sql)

jpnalvim avatar Nov 09 '23 20:11 jpnalvim

Hey @jpnalvim, As the error states, you should add -- depends_on: {{ ref('monitors_runs') }} at the top of your table1_.sql model. You can further read here on depends_on expression.

dapollak avatar Nov 21 '23 14:11 dapollak

@jpnalvim What is your version of dbt-core and dbt-elementary? We're getting pretty much the same error when we upgraded dbt version to 1.7.*, but with any previous dbt versions it ran smoothly and without any "depends_on" errors.

tautvydas-v avatar Feb 06 '24 11:02 tautvydas-v

Hi all, Closing this issue since it has no activity for over 3 months, but if it's still relevant please feel free to re-open!

Thanks, Itamar

haritamar avatar May 29 '24 00:05 haritamar