Andy Grove

[complex types] Improve "unsupported schema" message

2

### Describe the bug I tried testing complex type support from the Spark shell, by setting `COMET_PARQUET_SCAN_IMPL=native_datafusion` but I could not read a Parquet file containing structs and arrays. The...

### What is the problem the feature request solves? See comment in https://github.com/apache/datafusion-comet/pull/1620#discussion_r2035772596 for context ### Describe the potential solution _No response_ ### Additional context _No response_

Add support for native Parquet writes

3

### What is the problem the feature request solves? With ETL queries that read from Parquet, perform a transformation, and then write back out to Parquet, we currently have to...

[EPIC] Add support for all Map functions

3

### What is the problem the feature request solves? - [ ] [[ ] operator](https://docs.databricks.com/en/sql/language-manual/functions/bracketsign.html) - [ ] [map_concat function](https://docs.databricks.com/en/sql/language-manual/functions/map_concat.html) - [ ] [map_entries function](https://docs.databricks.com/en/sql/language-manual/functions/map_entries.html) - [ ] [map_filter function](https://docs.databricks.com/en/sql/language-manual/functions/map_filter.html)...

How can Comet be enabled by default without needing to configure memory?

3

### What is the problem the feature request solves? One of the challenges in having users switch to Comet is that it is typically necessary to specify how much memory...

### What is the problem the feature request solves? We would like Comet to fully support complex types (arrays, structs, and maps). This issue is for tracking all of the...

Add Comet to H2O.ai benchmark

2

### What is the problem the feature request solves? https://duckdblabs.github.io/db-benchmark/ ### Describe the potential solution _No response_ ### Additional context _No response_

### What is the problem the feature request solves? We recommend using off-heap memory when running Comet but all of our unit tests and the Spark SQL tests run with...

Stop running Rust tests in CI for all Java and Spark versions

1

### What is the problem the feature request solves? The rust tests do not use Java or Spark so we do not need to run them for different Java and...

### Describe the bug `ParquetEncryptionITCase` fails with `native_iceberg_compat`. ``` Error: ParquetEncryptionITCase>SparkFunSuite.run:69->SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run:69->AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run:1563->AnyFunSuite.runTests:1563->SparkFunSuite.runTest:69->SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest:69->SparkFunSuite.withFixture:227->SparkFunSuite.$anonfun$test$2:155->SparkFunSuite.failAfter:69->$anonfun$test$2:144->withSQLConf:44->org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf:44->$anonfun$test$3:144->$anonfun$new$1:55->$anonfun$new$2$adapted:55->$anonfun$new$2:56->withTempDir:44->org$apache$spark$sql$test$SQLTestUtils$$super$withTempDir:44->SparkFunSuite.withTempDir:245->$anonfun$new$3$adapted:56->$anonfun$new$3:62->withSQLConf:44->org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf:44->$anonfun$new$4:81->QueryTest.checkAnswer:159->QueryTest.checkAnswer:151 ? Spark Job aborted due to stage failure: Task 0 in stage 7.0 failed 1 times, most recent failure:...