polars icon indicating copy to clipboard operation
polars copied to clipboard

Is it possible to make self joins more able to be parallel without deadlock?

Open deanm0000 opened this issue 5 months ago • 2 comments

Description

I stumbled on this SO question and discovered this:

https://github.com/pola-rs/polars/blob/a6ee1e9dd1151a7d1aab390d45125e3c23425b14/crates/polars-lazy/src/physical_plan/planner/lp.rs#L479-L482

I wonder if there's any way to improve that check or workaround the deadlock issue such that more self joins can be parallel.

deanm0000 avatar Jan 25 '24 22:01 deanm0000