composable_kernel icon indicating copy to clipboard operation
composable_kernel copied to clipboard

Published 20 hours ago • verified publisher icon ROCmSoftwarePlatform

[CK_TILE] Add appendkv kernel to support mha with kvcache

Open poyenc opened this issue 7 months ago • 1 comments

Add new fmha_fwd_appendkv() API which runs ahead the fmha_fwd()/fmha_fwd_splitkv() API. The fmha_fwd_appendkv() + fmha_fwd()/fmha_fwd_splitkv() combination implement the functionality of mha_fwd_kvcache() in FA 2.5 (without paged-kvcache part)

poyenc avatar Jul 14 '24 22:07 poyenc