datafusion

datafusion copied to clipboard

Set the default value of `datafusion.execution.collect_statistics` to `true`

5

## Which issue does this PR close? - Closes ##16158 and #16444. ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested?...

AdamGS

Enable schema evolution for nested structs via adapt_column and custom adapter support in ListingTable

1

## Which issue does this PR close? This is the last of a series of PRs re-implementing #15295 to close #14757 by adding schema-evolution support for: - listing‐based tables -...

kosiew

## Which issue does this PR close? - Closes #16415. ## Rationale for this change The algorithm that is used for mark joins in `calculate_indices_by_join_type` might be confusing for newer...

jonathanc-n

Remove unused feature in `physical-plan` and fix compilation error in benchmark

2

## Which issue does this PR close? - Closes #16448. ## Rationale for this change #15913 introduced a feature to hid a test utilities module in `datafusion-physical-plan`. The module was...

AdamGS

Default to collecting statistics when creating LIstingTables

7

Today, when creating tables of parquet files using `CREATE EXTERNAL TABLE` or ListingTables, statistics are not gathered. This is good in the sense that creating the table is fast(er) but...

alamb

### Describe the bug Fuzzer failed during an unrelated change - https://github.com/apache/datafusion/actions/runs/15741542523/job/44367876525?pr=16449. Not sure how long GitHub retains logs, so I'm also adding the actual output: ``` ---- fuzz_cases::sort_query_fuzz::sort_query_fuzzer_runner stdout...

AdamGS

Fix constant window for evaluate stateful

5

## Which issue does this PR close? - Closes #16308. ## Rationale for this change I'm not very familiar with Comet, and I'm unsure how to verify this issue. So...

suibianwanwank

Blog post about TopK filter pushdown

4

### Is your feature request related to a problem or challenge? @adriangb has a great PR to add dynamic filtering with TopK in DataFusion here: - https://github.com/apache/datafusion/pull/15301 ### Describe the...

alamb

## Which issue does this PR close? ## Rationale for this change - While working on https://github.com/apache/datafusion/pull/16401 I noticed that the license header check ran twice: ![Screenshot 2025-06-18 at 4...

alamb

### Describe the bug One of the benchmark is behind a `bench` feature that isn't linted by default, so code that doesn't compile snuck into `main`. ### To Reproduce ```shell...

AdamGS