Zhuofu Chen comments

Repositories
Issues
Comments

Results 12 comments of


                                            Zhuofu Chen

[WIP][Feature] Support KV Partition for BatchPrefill kernel for Paged & Ragged KV-Cache.

@yzh119 Typing to ask if this is ready for use? I just find `BatchPrefillWithRaggedKVCacheDispatched` in main branch code but not sure if it could work.

QUESTION: How to implement a tree attention with flashinfer

Hi @yzh119 , thanks for your great contribution on this issue! I am willing to adopt flashinfer (w/ custom causal mask) in my current proj. However I got a small...