Can you support DeepSeek's inference acceleration? Thank you very much.

[joyhhheee]

[mrwyattii]

Hi @joyhhheee could you be a little more specific? Are you referring to supporting the models that DeepSeek has shared on HuggingFace?

[lynquantumman]

Run deepseek model on mii will generate repetitive results, don't if something got wrong

[brianyu-nexusflowai]

++ on models like https://huggingface.co/deepseek-ai/deepseek-coder-33b-instruct!

[ys2899]

The same problem here, I zoomed in and look at the codes why it generates repetitive results. I locate the problems over orig_module.forward = inference_container.module.forward in hybrid engine. Anybody has insights about this?