zhouheyun

Reproduce inference benchmark mentioned in the paper

4

I have a few questions about the inference efficiency of deepseek v2 1. > In order to efficiently deploy DeepSeek-V2 for service, we first convert its parameters into the precision...

[Feature Request] Any plan to support BF16 inference

2

Any plan to support BF16 inference? Our model encountered fp16 overflow after deployment.