detection_delay for volume anomalies tests with month scale

[Emmanuel2RPA]

Hello, sorry for the time between my slack message and the reporting of this bug. And I'll take the liberty of replying to my message here.

I wanted to tell you about a little problem I'm having with Elementary that I can't figure out, to see if other people have had the same experience. My use case is to exclude the actual month of the volume test, to do this, I want to use detection_delay with the following configuration:

# sources.yml
- name: my_table
  tests:
    - elementary.volume_anomalies:
        timestamp_column: calendar_day
        anomaly_direction: both
        anomaly_sensitivity: 2
        detection_delay:
          period: month
          count: 1

But I get the following error:

05:33:50    Compilation Error in test elementary_source_volume_anomalies_landing_zone_my_table_both__2__month__1__day__1__calendar_day (models/sources/sources.yml)
  detection_delay.period value should be one of ['hour', 'day', 'week'], got: month

  > in macro validate_detection_delay (macros/edr/tests/test_configuration/get_detection_delay.sql)
  > called by macro get_detection_delay (macros/edr/tests/test_configuration/get_detection_delay.sql)
  > called by macro get_anomalies_test_configuration (macros/edr/tests/test_configuration/get_anomalies_test_configuration.sql)
  > called by macro test_table_anomalies (macros/edr/tests/test_table_anomalies.sql)
  > called by macro test_volume_anomalies (macros/edr/tests/test_volume_anomalies.sql)
  > called by test elementary_source_volume_anomalies_landing_zone_my_table_both__2__month__1__day__1__calendar_day (models/sources/sources.yml)

I'm a little surprised by this error, because as you can see from the documentation, it's clearly described that this configuration can normally be applied on a monthly basis. In addition to this, we can see in the comments of this commit, that the modifications made should allow a use of the month. However, it can be seen that the detection delay values support the hour, day and week in the Elementary code : get_detection_delay.sql

And I've maybe an explanation to my problem, in get_buckets_configuration.sql used in test_table_anomalies > test_volume_anomalie, the developer used the following syntax sugar declaration to dynamically declare a time delta and this function doesn't seem to handle months natively class datetime.timedelta. The case was therefore handled for freshness tests, but not for volume tests.

[haritamar]

Hey @Emmanuel2RPA ! I believe you are right and month really isn't supported right now and the docs are wrong (were likely copied from the bucket configuration where we do support month).

We'll make sure to fix the docs, if you'd like by any chance to contribute support for month that will be awesome (as you said though, the implementation with timedelta will not work)