Andy Grove

Fix inconsistencies in operator configs for enabling/disabling/allowing incompatible

2

### What is the problem the feature request solves? Expressions have consistent configs for enabling, disabling, and allowing incompatible uses. This is because `CometExpressionSerde` has the following method: ```scala def...

[EPIC] [DISCUSS] Comet timezone handling

3

### What is the problem the feature request solves? Comet currently assumes that all native processing uses the UTC timezone. When reading from Parquet sources, Comet converts timestamps to UTC....

### What is the problem the feature request solves? Let's use this issue to coordinate on which issues/PRs should be included in the next release. Milestone https://github.com/apache/datafusion-comet/milestone/12 ### Describe the...

Add support for `RegExpExtract` and `RegExpExtractAll`

2

### What is the problem the feature request solves? Spark docs: - https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.regexp_extract.html - https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.regexp_extract_all.html ### Describe the potential solution _No response_ ### Additional context _No response_

### What is the problem the feature request solves? We currently have duplicate/overlapping logic in `CometExecRule` and `QueryPlanSerde` for determining whether operators are enabled or nor and whether we need...

Review use of hard-coded UTC assumptions

1

### What is the problem the feature request solves? This code pattern worries me. We should fall back to Spark if we do not know the timezone. ``` spark/src/main/spark-3.5/org/apache/comet/shims/CometExprShim.scala: .setTimezone(timeZoneId.getOrElse("UTC"))...

### What is the problem the feature request solves? Many (most?) Spark SQL tests do not start with Parquet scan, but use `RDDScanExec`, `LocalTableScanExec`, etc. so Comet falls back to...

Spark SQL test failures due to timestamp mismatch when LocalTableScan is native

1

### Describe the bug I saw this in Spark SQL tests with `spark.comet.sparkToColumnar.enabled=true`. ``` 2025-11-06T19:59:48.4097661Z [info] - sorting on TimestampType with nullable=true, sortOrder=List('a ASC NULLS FIRST) *** FAILED *** (72...

date_trunc incorrect results in non-UTC timezone

8

### Describe the bug PR https://github.com/apache/datafusion-comet/pull/2634 fixed some bugs with `trunc`/`date_trunc`, but I found another bug with the new tests added as part of this PR. The tests pass when...

### What is the problem the feature request solves? https://datafusion.apache.org/comet/user-guide/latest/compatibility.html has a list of compatible and incompatible cast operations but only shows primitive types. We should add documentation about casting...