spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-51671][SQL] Add column pruning ro Recursive CTEs

Open Pajaraja opened this issue 8 months ago • 0 comments

What changes were proposed in this pull request?

Modify ColumnPruning optimizer rule to successfully prune UnionLoops. For this, ColumnPruning had to migrated to inside the abstract Optimizer class.

Why are the changes needed?

To speed up rCTEs.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing + one new test in cte-recursion.sql.

Was this patch authored or co-authored using generative AI tooling?

No.

Pajaraja avatar Apr 25 '25 15:04 Pajaraja