datafusion-comet
datafusion-comet copied to clipboard
Re-enable tests for FIRST/LAST
What is the problem the feature request solves?
During the upgrade to DataFusion 47.0.0 () it was necessary to disable some tests that use FIRST and LAST because the behavior of these functions changed. These functions are non-determistic in Spark but previous versions of DataFusion had the same behavior as Spark for the single partition case used in the unit tests.
This issue is for re-enabling the tests somehow, or for implementing a Comet-specific implementation of FIRST/LAST to match Spark behavior (just for the single partition case used in tests).
These are the tests (all in CometAggregateSuite):
ignore("single group-by column + aggregate column, multiple batches, no null") {
ignore("multiple group-by columns + single aggregate column (first/last), with nulls") {
ignore("multiple group-by columns + multiple aggregate column (first/last), with nulls") {
ignore("all types first/last, with nulls") {
ignore("first/last") {
Describe the potential solution
No response
Additional context
No response