Andy Grove

[wip] feat: Add framework for supporting multiple telemetry providers

2

## Which issue does this PR close? Part of https://github.com/apache/datafusion-comet/issues/1718 ## Rationale for this change Experimenting with supporting OpenTelemetry as an option. ## What changes are included in this PR?...

chore: Improve reporting of fallback reasons for CollectLimit

1

## Which issue does this PR close? Part of https://github.com/apache/datafusion-comet/issues/1669 ## Rationale for this change I found it confusing to see `CollectLimitExec is not supported`, because I know that it...

### What is the problem the feature request solves? This epic is to track observability work to make it easier to monitor and debug issues with Comet in a distributed...

TPC-DS q67 causes OOM after repeated runs

1

### Describe the bug I am running with the following config: - SF 1000 (1TB) dataset - 32 executors - Each executor has 16 cores and 32 GB memory +...

[wip] Add scripts for running benchmarks on EC2

3

## Which issue does this PR close? Part of https://github.com/apache/datafusion-comet/issues/1636 ## Rationale for this change Make it easier for anyone to run the 100 GB benchmark on EC2 with local...

## Which issue does this PR close? Part of https://github.com/apache/datafusion-comet/issues/1704 ## Rationale for this change Demonstrate using an expression from the new `datafusion-spark` crate. ## What changes are included in...

Spark SQL test failures in native_iceberg_compat mode

6

### Describe the bug This issue is to track Spark SQL test failures in native_iceberg_compat mode. #### core1 ``` 2025-03-17T18:45:32.4547575Z [info] - SPARK-44185: relative LOCATION with Append SaveMode shall check...

Set up Comet + Iceberg integration tests in CI

1

### What is the problem the feature request solves? We want to avoid making code changes in Comet that cause regressions with the Iceberg integration. I would like to add...

Fix rat check errors during release process

1

### Describe the bug The rat exclude list needs updating to ignore these files: ``` NOT APPROVED: docs/source/_static/images/comet-dataflow.excalidraw (apache-datafusion-comet-0.8.0/docs/source/_static/images/comet-dataflow.excalidraw): false NOT APPROVED: native/core/testdata/backtrace.txt (apache-datafusion-comet-0.8.0/native/core/testdata/backtrace.txt): false NOT APPROVED: native/core/testdata/stacktrace.txt (apache-datafusion-comet-0.8.0/native/core/testdata/stacktrace.txt): false...

Benchmark automation

1

### What is the problem the feature request solves? I have been spending significant time manually running benchmarks, both during development and when preparing to release Comet. I have been...