datafusion

datafusion copied to clipboard

### Is your feature request related to a problem or challenge? As part of making the planner faster, we are updating the passes to avoid copying `LogicalPlan` and `Expr` (see...

alamb

Make repartitioning in `PhysicalPlan` output less confusing

18

### Is your feature request related to a problem or challenge? The use of multiple `RepartitionExec` and `CoalesceBatchesExec` I think makes the explain plans in DataFusion hard to read. This...

alamb

API in ParquetExec to pass in RowSelections to `ParquetExec` (enable custom indexes, finer grained pushdown)

13

### Is your feature request related to a problem or challenge? We are building / testing a specialized index for data stored in parquet that can tell us what row...

alamb

Display: Support `preserve_partitioning` on SortExec physical plan.

2

## Which issue does this PR close? Closes #10138 ## Rationale for this change `Display` of SortExec execution plan doesn't have visibility on `preserve_partitioning` field ``` SortExec: TopK(fetch=10000), expr=[NASME@0 ASC,VISITS@1...

kavirajk

Optimization: make the most of Hint::AcceptsSingular when call make_scalar_function to Improve performance

6

## Which issue does this PR close? Closes https://github.com/apache/arrow-datafusion/issues/10053 ## Rationale for this change ### Benchmark #### `overlay` function ```shell Gnuplot not found, using plotters backend 4args_with_3scalars/overlay/1024 time: [36.540 µs...

JasonLi-cn

Range/inequality joins are slow

10

### Describe the bug Joins where the `ON` filter are not equality, but rather inequalities like ` etc. seem slow. Atleast compared to DuckDB which seem like a direct "competitor"....

simonvandel

try to add local limit after the optimizer rule `join_selection`

3

### Is your feature request related to a problem or challenge? In our cases, we have left_table(multi file group) and right_table(single file group), and we write the sql ``` select...

liukun4515

CLI unable to execute queries with range(..) function

1

### Describe the bug Noticed when attempting to reproduce #9755 the initial query in that issue does not work when run against current main branch: ``` ❯ docker build -f...

Omega359

LRU DashMap to cache objectMeta

1

## Which issue does this PR close? part of #9964 Closes #. ## Rationale for this change ## What changes are included in this PR? ## Are these changes tested?...

Lordworms

Add benchmark for DFSchema

1

## Which issue does this PR close? Related to #7698. ## Rationale for this change To compare approaches of DFSchema optimization. ## What changes are included in this PR? Add...

oleggator