Daft
Daft copied to clipboard
refactor: convert scan task `split_by_row_groups` and `merge_by_sizes` into optimizer rule
Is your feature request related to a problem? Please describe.
Currently there is an optimization inlined inside translate_single_logical_node.
I think it'd make the code a bit easier to reason about if this was instead extracted into an optimizer pass.
Describe the solution you'd like
split_by_row_groups and merge_by_sizes is moved out of translate_single_logical_node
and instead moved into a dedicated optimizer rule
Describe alternatives you've considered Keep the code as is. It technically works, but it does make the code a bit more difficult to reason about.