elementary icon indicating copy to clipboard operation
elementary copied to clipboard

Feature request: include test name in `data_monitoring_metrics` and `metrics_anomaly_score`

Open garfieldthesam opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. My company's data quality strategy revolves around test names (not the name of the generic test, but a custom name created for each test that makes it easy for the on-call person to understand what the test means). For example, this elementary test is named low_table_name_column_name_volume_anomaly.

      - name: low_table_name_column_name_volume_anomaly
        dimensions:
          - column_name
        timestamp_column: datestamp
        backfill_days: 3 
        seasonality: day_of_week
        anomaly_direction: both   
        anomaly_sensitivity: 2
        test_name: elementary.dimension_anomalies
        severity: warn   
        error_if: '>4'   
        warn_if: '>2'  

It's difficult to set up tools to help on-calls analyze failures/warnings because data_monitoring_metrics and metrics_anomaly_score do not have the test name.

It would also make the package more generally usable because the test name is what's reported in the dbt job logs.

Describe the solution you'd like

Add column test_name to data_monitoring_metrics and metrics_anomaly_score

Describe alternatives you've considered I will probably base the tooling on the combination of fields that are available in data_monitoring_metrics and metrics_anomaly_score but it will be cumbersome and we will update it as soon as possible if/when this data is added to data_monitoring_metrics and metrics_anomaly_score

Additional context N/A

Would you be willing to contribute this feature? Not any time in the foreseeable future

garfieldthesam avatar Jan 09 '24 21:01 garfieldthesam