executorch icon indicating copy to clipboard operation
executorch copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

how to realize the sliding window of kv cache?

Open l2002924700 opened this issue 6 months ago • 0 comments

hello, now I want to realize the sliding window of kv cache, so dynamic allocation and reclamation of memory needs to be realized. could you please teach me how to realize the dynamic allocation and reclamation of memory in the transformer? Thank you in advanced.

l2002924700 avatar Aug 07 '24 07:08 l2002924700