paged-attention-minimal icon indicating copy to clipboard operation
paged-attention-minimal copied to clipboard
verified publisher icon tspeterkim

The update of k_cache_paged and v_cache_paged

Open NonvolatileMemory opened this issue 1 year ago • 1 comments

Hi tspeterkim,

Thanks for your great code!

I am wondering why the v_cache_paged and k_cache_paged is not updated.

NonvolatileMemory avatar Sep 16 '24 12:09 NonvolatileMemory

flash_attn_with_kv_cache will automatically update, close the issue

NonvolatileMemory avatar Sep 16 '24 14:09 NonvolatileMemory