MNN icon indicating copy to clipboard operation
MNN copied to clipboard
verified publisher icon alibaba

Will MNN Chat app support ”Speculative Decoding“？

Open BeetSoup128 opened this issue 7 months ago • 1 comments

请问是否有计划下一步加入Speculative Decoding？MNN框架下的LLM拥有高度的一致性，相对大内存与较低的算力更适合双模型同时加载。

BeetSoup128 avatar May 01 '25 11:05 BeetSoup128

正在实现中，预计本月会支持

jxt1234 avatar May 03 '25 05:05 jxt1234

Marking as stale. No activity in 60 days.

github-actions[bot] avatar Jul 02 '25 09:07 github-actions[bot]