Feat: bitmap_count impl and some refactoring
Which issue does this PR close?
Part of: https://github.com/apache/datafusion-comet/issues/2443
Rationale for this change
For the bitmap_count and read_side_padding functions, Spark uses a StaticInvoke wrapper. (https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/bitmapExpressions.scala#L85)
What changes are included in this PR?
- statics.scala - CometStaticInvoke wrapper has been added for the comet functions: bitmap_count, read_side_padding
- QueryPlanSerde.scala - removed unused matching cases
- CometBitmapExpressionSuite.scala - new UT
How are these changes tested?
A new UT has been added.
Codecov Report
:x: Patch coverage is 56.25000% with 7 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 58.74%. Comparing base (f09f8af) to head (728b7c9).
:warning: Report is 581 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| ...rc/main/scala/org/apache/comet/serde/statics.scala | 50.00% | 6 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #2518 +/- ##
============================================
+ Coverage 56.12% 58.74% +2.61%
- Complexity 976 1452 +476
============================================
Files 119 148 +29
Lines 11743 13642 +1899
Branches 2251 2362 +111
============================================
+ Hits 6591 8014 +1423
- Misses 4012 4405 +393
- Partials 1140 1223 +83
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
Starting CI
Thanks @comphead, but the test failures occurred due to DataFusion's lack of support for dictionary-encoded types.
Cause: org.apache.comet.CometNativeException: Error from DataFusion: bitmap_count expects Binary/BinaryView/FixedSizeBinary/LargeBinary as argument, got Dictionary(Int32, Binary).
Created a task: https://github.com/apache/datafusion/issues/18058