mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

Published 20 hours ago • verified publisher icon EricLBuehler

Initial KV RingAttention code

Open joshpopelka20 opened this issue 6 months ago • 6 comments

This is the start of the RingAttention code. The changes so far have been to create multiple KV caches (if multiple num_devices) and to try to create separate chunks.

joshpopelka20 avatar Aug 14 '24 20:08 joshpopelka20