datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Stop copying LogicalPlan and Exprs in `CommonSubexprEliminate`

Open alamb opened this issue 1 year ago • 3 comments

Is your feature request related to a problem or challenge?

As part of making the planner faster, we are updating the passes to avoid copying LogicalPlan and Expr (see https://github.com/apache/datafusion/issues/9637)

Recent profiling (https://github.com/apache/datafusion/issues/9637#issuecomment-2073325456) shows that a non trivial amount of plan time happens in CommonSubexprEliminate

Describe the solution you'd like

I would like to reduce the amount of copying in this pass that it no longer appears in profiling traces

Describe alternatives you've considered

No response

Additional context

No response

alamb avatar Apr 24 '24 11:04 alamb

I believe @peter-toth is working on this one

alamb avatar Apr 24 '24 11:04 alamb

I am, but how this issue relates to https://github.com/apache/datafusion/issues/9873? Are they duplicates?

peter-toth avatar Apr 24 '24 12:04 peter-toth

I am, but how this issue relates to #9873? Are they duplicates?

Good call -- I think they are dupes. Closing this one

alamb avatar Apr 24 '24 13:04 alamb

@alamb, why did you reopen this?

peter-toth avatar May 01 '24 17:05 peter-toth

@alamb, why did you reopen this?

I am not sure. Sorry about that

alamb avatar May 01 '24 20:05 alamb