[FEAT] Ray shuffle experiment

[jaychia]

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #2883 will not alter performance

_{Comparing jay/shuffle-experiment (ade5e2f) with main (768f422)}

Summary

✅ 17 untouched benchmarks

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 24.48037% with 327 lines in your changes missing coverage. Please review.

Project coverage is 78.38%. Comparing base (868bbfa) to head (ade5e2f). Report is 85 commits behind head on main.

Files with missing lines	Patch %	Lines
daft/runners/ray_runner.py	18.22%	166 Missing :warning:
daft/execution/physical_plan.py	3.50%	55 Missing :warning:
src/daft-scheduler/src/scheduler.rs	0.00%	34 Missing :warning:
src/daft-plan/src/physical_planner/translate.rs	57.77%	19 Missing :warning:
src/daft-plan/src/physical_ops/exchange_op.rs	0.00%	16 Missing :warning:
daft/execution/physical_plan_shuffles.py	74.46%	12 Missing :warning:
...sical_optimization/rules/reorder_partition_keys.rs	0.00%	12 Missing :warning:
src/daft-plan/src/physical_plan.rs	0.00%	9 Missing :warning:
daft/context.py	33.33%	2 Missing :warning:
daft/runners/pyrunner.py	66.66%	1 Missing :warning:
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2883      +/-   ##
==========================================
- Coverage   78.71%   78.38%   -0.34%     
==========================================
  Files         597      599       +2     
  Lines       69735    70131     +396     
==========================================
+ Hits        54895    54969      +74     
- Misses      14840    15162     +322     

Flag	Coverage Δ
	78.38% <24.48%> (-0.34%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
daft/runners/runner.py	73.52% <100.00%> (+1.65%)	:arrow_up:
src/daft-plan/src/physical_ops/mod.rs	0.00% <ø> (ø)
daft/runners/pyrunner.py	85.07% <66.66%> (-0.18%)	:arrow_down:
src/daft-plan/src/display.rs	80.00% <0.00%> (-0.33%)	:arrow_down:
daft/context.py	78.61% <33.33%> (+1.05%)	:arrow_up:
src/daft-plan/src/physical_plan.rs	51.21% <0.00%> (-1.38%)	:arrow_down:
daft/execution/physical_plan_shuffles.py	74.46% <74.46%> (ø)
...sical_optimization/rules/reorder_partition_keys.rs	75.25% <0.00%> (-4.97%)	:arrow_down:
src/daft-plan/src/physical_ops/exchange_op.rs	0.00% <0.00%> (ø)
src/daft-plan/src/physical_planner/translate.rs	91.97% <57.77%> (-1.88%)	:arrow_down:
... and 3 more

... and 4 files with indirect coverage changes

[samster25]

@jaychia is PR still relevant after merging in your other one?

[jaychia]

@jaychia is PR still relevant after merging in your other one?

No, this is just used by @colin-ho for reference. Closing.