Cody Yu comments

Repositories
Issues
Comments

Results 161 comments of


                                            Cody Yu

[RFC]: Offload KV cache to CPU in V1

Thanks for the RFC. In general I agree that KV cache offloading could be useful in certain scenarios, and the proposed approach looks reasonable to me. Meanwhile, please add the...