ggml icon indicating copy to clipboard operation
ggml copied to clipboard

Published 20 hours ago • verified publisher icon ggerganov

Why perform such operations on k and v as shown in the above diagram?

Open EveningLin opened this issue 11 months ago • 1 comments

image Why perform such operations on k and v as shown in the above diagram?

EveningLin avatar Mar 16 '24 07:03 EveningLin

In order to copy and persist the current key and values (Kcur and Vcur) to the kv cache.

YavorGIvanov avatar Mar 19 '24 11:03 YavorGIvanov