delta icon indicating copy to clipboard operation
delta copied to clipboard

Published 20 hours ago • verified publisher icon delta-io

[KERNEL] CollatedPredicate support

Open ilicmarkodb opened this issue 7 months ago • 1 comments

Which Delta project/connector is this regarding?

  • [ ] Spark
  • [ ] Standalone
  • [ ] Flink
  • [X] Kernel
  • [ ] Other (fill in here)

Description

This PR introduces CollatedPredicate as a new type of Predicate for comparing strings in collated fashion. The DefaultEngine will only support the default collation (SPARK.UTF8_BINARY), meaning ExpressionHandler should be implemented for using any other collation.

How was this patch tested?

Tests added to CollatedPredicateSuite and DefaultExpressionEvaluatorSuite. Tests for file pruning will be added in the next PR, as modifying DataSkippingPredicate is required. Currently, DataSkippingPredicate is just a regular Predicate, but it needs to be updated to also include information about collation.

Does this PR introduce any user-facing changes?

No.

ilicmarkodb avatar Mar 10 '25 16:03 ilicmarkodb