MiniCPM-o icon indicating copy to clipboard operation
MiniCPM-o copied to clipboard

Published 20 hours ago • verified publisher icon OpenBMB

int4和bffloat16推理时间问题（着急）

Open githublsk opened this issue 9 months ago • 2 comments

用如下代码分别测试MiniCPM-2B-dpo-bf16和MiniCPM-dpo-Int4两个模型，推理时间MiniCPM-2B-dpo-bf16有3秒多，MiniCPM-dpo-Int4有10秒以上，请问原因是啥？ image

githublsk avatar May 22 '24 08:05 githublsk