elementary
elementary copied to clipboard
`Invalid Date` columns from Snowflake break elementary
Describe the bug
I got some weird errors when enabling elementary, crashing dbt test
with a segmentation fault. I narrowed this down to being caused by some snowflake date columns with Invalid Date
in them. I cannot provide a concise test to reproduce the segmentation fault, but the below causes dbt to crash when elementary is enabled, it runs fine without. I think the problem is related.
To Reproduce Steps to reproduce the behavior:
- Run below test in Snowflake environment with elementary enabled
SELECT *
FROM (VALUES
(1620502461213752::timestamp_ntz, 1234),
(1620502461213752::timestamp_ntz, 34234),
(1620502461213752::timestamp_ntz, 1234),
(1620502461213752::timestamp_ntz, 34234),
(1620502461213752::timestamp_ntz, 1234),
(1620502461213752::timestamp_ntz, 34234)
) AS fake_rows (crap_date, xxxx)
- Should get the following output or similar:
12:02:53 Running 1 on-run-start hook
12:02:57 1 of 1 START hook: elementary.on-run-start.0 ................................... [RUN]
12:02:57 1 of 1 OK hook: elementary.on-run-start.0 ...................................... [OK in 0.00s]
12:02:57
12:02:58 Concurrency: 4 threads (target='snowflake-okta')
12:02:58
12:02:58 1 of 1 START test elementary_failure ........................................... [RUN]
12:03:02 Unhandled error while executing test.caspian.elementary_failure
252005: Failed to convert current row, cause: year 51353648 is out of range
12:03:02 1 of 1 ERROR elementary_failure ................................................ [ERROR in 3.86s]
12:03:02
12:03:02 Running 1 on-run-end hook
12:03:21 1 of 1 START hook: elementary.on-run-end.0 ..................................... [RUN]
12:03:21 1 of 1 OK hook: elementary.on-run-end.0 ........................................ [OK in 0.00s]
12:03:21
12:03:21
12:03:21 Finished running 1 test, 2 hooks in 0 hours 0 minutes and 41.17 seconds (41.17s).
12:03:21
12:03:21 Completed with 1 error and 0 warnings:
12:03:21
12:03:21 252005: Failed to convert current row, cause: year 51353648 is out of range
- Disable elementary, you should get a regular test failure
Expected behavior Normal test failures. Screenshots If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
- edr Version: 0.10.0
- dbt package Version: 0.10.3 Additional context attaching dbt.log of where i got the segmentation fault. + here's the thread on slack https://elementary-community.slack.com/archives/C02CTC89LAX/p1695999665254459 dbt.log