polars
polars copied to clipboard
Is it possible to make self joins more able to be parallel without deadlock?
Description
I stumbled on this SO question and discovered this:
https://github.com/pola-rs/polars/blob/a6ee1e9dd1151a7d1aab390d45125e3c23425b14/crates/polars-lazy/src/physical_plan/planner/lp.rs#L479-L482
I wonder if there's any way to improve that check or workaround the deadlock issue such that more self joins can be parallel.