Andy Grove

We do not respect ignoreNulls in first_value / last_value aggregates

3

### Describe the bug When creating the native plan for `first_value` / `last_value` aggregates, we currently hard-code `ignoreNulls` instead of using the value from the protobuf encoding: ```rust AggregateExprBuilder::new(Arc::new(func), vec![child])...

### What is the problem the feature request solves? I would like to have some version of the fuzz testing tool running in CI. Initial thoughts on tasks: - Implement...

### Describe the bug I added a new test in https://github.com/apache/datafusion-comet/pull/1308 and it fails on Spark 4.0 with: ``` 2025-01-20T02:01:13.8889203Z - array_remove - fallback for unsupported type struct *** FAILED...

### Describe the bug ``` - test with low buffer spill threshold *** FAILED *** (224 milliseconds) 2025-01-09T12:57:50.2870347Z java.lang.AssertionError: assertion failed: expected test with low buffer spill threshold to spill,...

### Describe the bug When we fix https://github.com/apache/datafusion-comet/issues/1248 we find a new bug that causes the following Spark SQL test to fail: ``` - subquery/exists-subquery/exists-orderby-limit.sql *** FAILED *** (1 second,...

### Describe the bug When we fix https://github.com/apache/datafusion-comet/issues/1248 we find a new bug that causes the following Spark SQL test to fail: ``` - write null values *** FAILED ***...

### Describe the bug When we fix https://github.com/apache/datafusion-comet/issues/1248 we find a new bug that causes the following Spark SQL test to fail: - `reverse preceding/following range between with aggregation` ###...

count distinct on NaN produces incorrect results

7

### Describe the bug In https://github.com/apache/datafusion-comet/pull/1209 we now fall back to columnar shuffle in some cases where native shuffle is not supported, rather than just falling back to Spark. This...

### What is the problem the feature request solves? We track encoding metrics for native shuffle but not yet for columnar shuffle:  ### Describe the potential solution _No response_...

### What is the problem the feature request solves? I get better performance with larger batch sizes in shuffle writer (at the expense of using more memory). It would be...