Spilling to host memory seamlessly

[madsbk]

This is based on #10746 and uses the newly refactored Buffer.

We introduce a new deivce-buffer-like class, SpillableBuffer, that will spill its device memory to host memory if RMM is running out of unused memory.

In order to enable this new spilling feature, set the environment variable CUDF_SPILL=on, which make cuDF use SpillableBuffer buffers for most of its allocations.

Closes #10746

Checklist

• [x] New or existing tests cover these changes.
• [x] Avoid changes to libcudf
• [ ] The documentation is up to date with these changes.

[codecov[bot]]

Codecov Report

Base: 88.11% // Head: 88.25% // Increases project coverage by +0.14% :tada:

Coverage data is based on head (a3ad5a6) compared to base (5c2150e). Patch coverage: 88.55% of modified lines in pull request are covered.

:exclamation: Current head a3ad5a6 differs from pull request most recent head e31cc6a. Consider uploading reports for the commit e31cc6a to get more accurate results

Additional details and impacted files

@@               Coverage Diff                @@
##           branch-22.12   #11553      +/-   ##
================================================
+ Coverage         88.11%   88.25%   +0.14%     
================================================
  Files               133      137       +4     
  Lines             21982    22487     +505     
================================================
+ Hits              19369    19846     +477     
- Misses             2613     2641      +28     

Impacted Files	Coverage Δ
python/cudf/cudf/core/column/decimal.py	90.60% <ø> (ø)
python/cudf/cudf/core/column/numerical.py	95.49% <ø> (ø)
python/cudf/cudf/core/buffer/spill_manager.py	76.59% <76.59%> (ø)
python/cudf/cudf/core/column/column.py	88.46% <80.00%> (ø)
python/cudf/cudf/core/buffer/spillable_buffer.py	93.45% <93.45%> (ø)
python/cudf/cudf/core/dtypes.py	96.64% <93.75%> (+0.11%)	:arrow_up:
python/cudf/cudf/core/buffer/utils.py	96.87% <96.87%> (ø)
python/cudf/cudf/core/buffer/buffer.py	91.40% <96.96%> (ø)
python/cudf/cudf/core/abc.py	94.44% <100.00%> (+8.08%)	:arrow_up:
python/cudf/cudf/core/buffer/__init__.py	100.00% <100.00%> (ø)
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[quasiben]

Should we close this if it is superseded by https://github.com/rapidsai/cudf/pull/12106 ?

[vyasr]

Should we close this if it is superseded by #12106 ?

I don't mind either way. This branch is definitely still relevant. There is code for logging spilling statistics in this PR that didn't make it into #12106. As long as we keep the branch around I'm indifferent to what we do with this PR in the short term and would defer to whatever @madsbk finds most convenient.

[jakirkham]

FWIW it is possible to checkout a PR locally even after it is closed and the branch that sent it is deleted.