OmniQuant icon indicating copy to clipboard operation
OmniQuant copied to clipboard
verified publisher icon OpenGVLab

[New Feature] Seek MLA Supported by Smooth

Open RanchiZhao opened this issue 1 year ago • 0 comments

Will MLA that used in DeepSeek-V2 (https://huggingface.co/deepseek-ai/DeepSeek-V2-Chat and https://arxiv.org/abs/2405.04434) be supported by activation smooth method?

RanchiZhao avatar Jul 10 '24 08:07 RanchiZhao