kvcached icon indicating copy to clipboard operation
kvcached copied to clipboard
Published 4 weeks ago • verified publisher icon ovg-project

[TODO] Support kvcached offloading to other storage like CPU memory

Open jiarong0907 opened this issue 3 months ago • 0 comments

When the GPU memory is almost full, kvcached can support offloading KV cache to CPU memory or even disks. Do this using CUDA UVM or more application semantics?

jiarong0907 avatar Aug 31 '25 22:08 jiarong0907