datafusion

datafusion copied to clipboard

[EPIC] Stop copying `LogicalPlan` during OptimizerPasses

47

### Is your feature request related to a problem or challenge? Broken out from https://github.com/apache/arrow-datafusion/issues/9577 where @mustafasrepo @comphead and @jayzhan211 and I were discussing optimizer performance TLDR is that the...

alamb

Any plan to support JSON or JSONB?

31

### Is your feature request related to a problem or challenge? Datafusion does not support JSON/JSONB datatype. Is there a plan to support it in the future? ### Describe the...

dojiong

`select array_concat([])` panicked

8

### Describe the bug thread 'main' panicked at datafusion/datafusion/functions-array/src/concat.rs:275:32: index out of bounds: the len is 0 but the index is 0 ### To Reproduce Run any of the following...

jonahgao

WIP -- still has many more copies to remove ## Which issue does this PR close? Part of https://github.com/apache/arrow-datafusion/issues/9637 related to https://github.com/apache/datafusion/issues/9873 ## Rationale for this change Let's make planning...

alamb

Stop copying `Expr`s and LogicalPlans so much during Common Subexpression Elimination

16

### Is your feature request related to a problem or challenge? The common subexpression elimination pass copies many `Expr`s around. You can see this performance impact this has by looking...

alamb

WIP: directly pass Vec<Expr> instead of &[Expr] to avoid clone

1

## Which issue does this PR close? change performance of eliminate_subexpr. Still in progress, this is just for benchmark test part of #9873 ## Rationale for this change ## What...

Lordworms

Stop copying LogicalPlan and Exprs in `CommonSubexprEliminate`

3

### Is your feature request related to a problem or challenge? As part of making the planner faster, we are updating the passes to avoid copying `LogicalPlan` and `Expr` (see...

alamb

"Cannot infer common argument type for comparison operation Union..."

5

### Describe the bug See https://github.com/datafusion-contrib/datafusion-functions-json/pull/3 I have a union defined by ```rs DataType::Union( UnionFields::new( vec![0, 1, 2, 3, 4, 5, 6], vec![ Field::new("null", DataType::Boolean, true), Field::new("bool", DataType::Boolean, false), Field::new("int",...

samuelcolvin

Avoid copies in `TypeCoercion` via TreeNode API

1

Draft as it has a failure in tpchds planning for some reason ## Which issue does this PR close? Closes https://github.com/apache/datafusion/issues/10210 Part of https://github.com/apache/arrow-datafusion/issues/9637 -- let's make DataFusion planning faster...

alamb

### Is your feature request related to a problem or challenge? As part of making the planner faster, we are updating the passes to avoid copying `LogicalPlan` and `Expr` (see...

alamb