Results
11
comments of
Ming Wei
Thanks for the elaboration. I don't think TRTLLM supports such use cases. A workaround that you can try is to pad zeroes to q/k/v to make their dimension match, however...