elementary
elementary copied to clipboard
Data test broken when switching to materialized views on postgres
Describe the bug
On postgres, when using elementary.column_anomalies test on a column that is part of a table using materialized views, an "Unable to find column" error occurs during the elemenatary data test run.
To Reproduce
Create a table using materialized view materialization:
{{ config(materialized='materialized_view') }}
SELECT bill_usd FROM table
Configure test in DBT table spec:
models:
- name: table
columns:
- name: bill_usd
tests:
- elementary.column_anomalies:
detection_period:
period: day
count: 2
training_period:
period: day
count: 35
detection_delay:
period: day
count: 5
column_anomalies:
- sum
config:
severity: warn
Run elementary:
2024-08-27 12:05:46,796 | INFO | root | 00:05:42 Compilation Error in test ...bill_usd__day__5__day__2__day__35 (models/....yml)
2024-08-27 12:05:46,796 | INFO | root | Unable to find column `bill_usd` in `....TABLE`
2024-08-27 12:05:46,796 | INFO | root |
2024-08-27 12:05:46,796 | INFO | root | > in macro test_column_anomalies (macros/edr/tests/test_column_anomalies.sql)
2024-08-27 12:05:46,797 | INFO | root | > called by test ...bill_usd__day__5__day__2__day__35 (...)
This problem goes away when I remove the materialization, or switch to table materialization.
Expected behavior
I expect data tests to continue working when I switch to materialized_view materialization.
Screenshots
n/a
Environment (please complete the following information):
- Elementary CLI (edr) version: 0.16.1
- Elementary dbt package version: 0.16.0
- dbt version you're using: 1.8.5
- Data warehouse: postgres
- Infrastructure details (e.g. operating system, prod / dev / staging, deployment infra, CI system, etc): freshly built docker image
prefecthq/prefect:2-python3.9