datafusion

datafusion copied to clipboard

remove Duplication between `ParquetFormat` and `ParquetReadOptions` is confusing

2

Draft as it builds on https://github.com/apache/arrow-datafusion/pull/2985 # Which issue does this PR close? Closes https://github.com/apache/arrow-datafusion/issues/2987 # Rationale for this change The duplication was confusing. However, I am not sure about...

alamb

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** While working on https://github.com/apache/arrow-datafusion/pull/2985 I noticed I had to repeat the `skip_metadata`...

alamb

See https://github.com/apache/arrow-datafusion/pull/2966/files#r930366992 for more details

liukun4515

I'd love to create a pull request if there isn't any problem. https://github.com/apache/arrow-datafusion/compare/master...thomas-k-cameron:arrow-datafusion:stat-should-be-None-instead-of-Some(0) **Describe the bug** Some fields of `Statistics` could return `Some(0)` when the value is not available. TODO...

thomas-k-cameron

WIP: Starting point for optimizer integration tests

4

# Which issue does this PR close? Closes https://github.com/apache/arrow-datafusion/issues/2911 # Rationale for this change I would like integration tests in the optimizer crate where we can easily add SQL and...

andygrove

Better not to display partitions info for ParquetExec

4

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Suppose there're tens of thousands of files needs to be scanned for...

yahoNanJing

Datafusion Optimizer Enhancement

2

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Enhance Optimizer Summary: - [x] https://github.com/apache/arrow-datafusion/issues/217 - [ ] https://github.com/apache/arrow-datafusion/issues/475 - [...

jackwener

[EPIC] Support all TPC-H queries in benchmark

2

*Note*: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-11519 This is a tracking issue for supporting all of the TPC-H queries. Supporting all of the queries will add a number of features and...

alamb

Migrate the `cross join -> inner join optimization` from the planner to the optimizer

1

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.** Migrate the cross join -> inner join optimization from the planner to...

xudong963

Epic: Complete ROW Format (Missing features)

1

**Goal: a complete row implementation, fully used in pipeline breaker operators when possible.** **Summary** TLDR: The key focus of this work is to speed up fundamentally row oriented operations like...

yjshen