dash-infer icon indicating copy to clipboard operation
dash-infer copied to clipboard

Published 20 hours ago • verified publisher icon modelscope

CPU specifications for Qwen2-7B models

Open LiweiPE opened this issue 6 months ago • 1 comments

I would like to know what should it be the minimum requirement specifications for running 7B models. I have this configuration and only can run 1.5B with average throughput 8.1 token/s. image image image

LiweiPE avatar Aug 12 '24 07:08 LiweiPE