splink icon indicating copy to clipboard operation
splink copied to clipboard

Consistent date difference level across backends

Open ADBond opened this issue 1 year ago • 1 comments

Datediff measures boundary partitions, so e.g. '30th Jan to 1st Feb is 1 month, but so is 1st Jan to 28th Feb'. Currently there is discrepancy across backends in DatediffLevel which do things like this (duckdb), and those which treat year/month as 'time intervals' (spark, postgres).

Arguably the 'boundary partition' approach is not that useful compared to 'fixed intervals', which is perhaps more intuitive.

Proposal is to replace DatediffLevel with a DateDifferenceLevel (changing name to make it clear this does not align with 'datediff' semantics), which behaves consistently across backends, and essentially has 1 year == 12 months == 30 days (modulo some details on those specific numbers). True datediff behaviour using boundary partitions will still be available using custom SQL for any users that may still require this.

ADBond avatar Jan 12 '24 13:01 ADBond

Yep - fully support this, and should make the implementation simpler as well!

RobinL avatar Jan 16 '24 10:01 RobinL

Closed by #1940

ADBond avatar Jul 25 '24 07:07 ADBond