dbt-utils icon indicating copy to clipboard operation
dbt-utils copied to clipboard

Recency test to fail when no recent records are found

Open marzaccaro opened this issue 3 months ago • 0 comments

resolves #1041

Problem

The dbt_utils.recency test incorrectly passes when a where clause filters out all records.

Current behavior: When a where clause results in no matching records, most_recent becomes NULL. The test only checks where most_recent < {{ threshold }}, but since NULL < threshold evaluates to NULL (not true), no rows are returned and the test incorrectly passes.

Expected behavior: The test should fail when no recent data exists (even when filtered by a where clause), indicating missing recent data.

Solution

Modified the recency macro in macros/generic_tests/recency.sql to include or most_recent is null in the WHERE clause. This ensures that when a where clause filters out all records (causing most_recent to be null), the test correctly fails by returning the null row.

Changes made:

  • Updated the WHERE condition from where most_recent < {{ threshold }} to where most_recent < {{ threshold }} or most_recent is null
  • Added integration test recency_with_where_filter that reproduces the original bug and verifies the fix
  • Test uses data with old timestamps (10-15 days) and a where clause that filters to non-existent records

Testing approach:

  • Before fix: Test incorrectly passes (returns 0 rows)
  • After fix: Test correctly fails (returns 1 row with null most_recent)
  • Verified existing recency tests continue to work as expected

Checklist

  • [x] This code is associated with an issue which has been triaged and [accepted for development](https://docs.getdbt.com/docs/contributin g/oss-expectations#pull-requests).
  • [x] I have read the contributing guide and understand what's expected of me
  • [x] I have run this code in development and it appears to resolve the stated issue
  • [x] This PR includes tests, or tests are not required/relevant for this PR
  • [x] I have updated the README.md (if applicable)

marzaccaro avatar Sep 24 '25 14:09 marzaccaro