spark icon indicating copy to clipboard operation
spark copied to clipboard

[SQL][MINOR] Defensive code for number of Partitions in rCTEs

Open Pajaraja opened this issue 1 month ago • 1 comments

What changes were proposed in this pull request?

Set the number of partitions in recursive CTEs to be at least 1.

Why are the changes needed?

Defensive programming to avoid zero-partition RDDs, which could cause potential issues in downstream operations like coalesce.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing CIs.

Was this patch authored or co-authored using generative AI tooling?

No.

Pajaraja avatar Dec 01 '25 12:12 Pajaraja

Is it possible to add a unit test by building physical plans directly? e.g. RDDScanExec can host a 0-partition RDD.

cloud-fan avatar Dec 02 '25 00:12 cloud-fan