Recency test to fail when no recent records are found
resolves #1041
Problem
The dbt_utils.recency test incorrectly passes when a where clause filters
out all records.
Current behavior: When a where clause results in no matching records,
most_recent becomes NULL. The test only checks where most_recent < {{ threshold }}, but since NULL < threshold evaluates to NULL (not true), no
rows are returned and the test incorrectly passes.
Expected behavior: The test should fail when no recent data exists (even when filtered by a where clause), indicating missing recent data.
Solution
Modified the recency macro in macros/generic_tests/recency.sql to include
or most_recent is null in the WHERE clause. This ensures that when a where
clause filters out all records (causing most_recent to be null), the test
correctly fails by returning the null row.
Changes made:
- Updated the WHERE condition from
where most_recent < {{ threshold }}towhere most_recent < {{ threshold }} or most_recent is null - Added integration test
recency_with_where_filterthat reproduces the original bug and verifies the fix - Test uses data with old timestamps (10-15 days) and a where clause that filters to non-existent records
Testing approach:
- Before fix: Test incorrectly passes (returns 0 rows)
- After fix: Test correctly fails (returns 1 row with null
most_recent) - Verified existing recency tests continue to work as expected
Checklist
- [x] This code is associated with an issue which has been triaged and [accepted for development](https://docs.getdbt.com/docs/contributin g/oss-expectations#pull-requests).
- [x] I have read the contributing guide and understand what's expected of me
- [x] I have run this code in development and it appears to resolve the stated issue
- [x] This PR includes tests, or tests are not required/relevant for this PR
- [x] I have updated the README.md (if applicable)