dask-cuda
dask-cuda copied to clipboard
Benchmark Spilling
We should build a benchmark to profile spilling in at least the following scenarios:
- performance: measure that our spilling is efficient in a larger workflow -- maybe this is the merge benchmark with larger than memory data
- stability: ensure that we never OOM
We should also test these scenarios with both the exist spilling infrastructure and the RAPIDS Spilling Manager
Briefly discussed with @wence- and @charlesbluca
This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.