datafusion
datafusion copied to clipboard
Stop copying LogicalPlan and Exprs in `PushDownFilter`
Is your feature request related to a problem or challenge?
Part of https://github.com/apache/datafusion/issues/9637
As part of making the planner faster, we are updating the optimizer passes to avoid copying LogicalPlan and Expr (see https://github.com/apache/datafusion/issues/9637)
Describe the solution you'd like
I would like to reduce the amount of copying in this pass (even though it doesn't appear in current profiling)
Describe alternatives you've considered
Apply the model from @Lordworms in https://github.com/apache/datafusion/pull/10166 to this pass 2. Update OptimizerRule::supports_rewrite` to return true
- Update
OptimizerRuleto userewrite - Update the pass itself to not copy the LogicalPlan (ideally using the
TreeNodeAPI) - it is implemented forLogicalPlan(API) andExpr(API)
Other examples: https://github.com/apache/datafusion/pull/10218
Additional context
I am starting to unravel the remaining copies in PushDownFilter -- it is non trivial