GPTQ-for-LLaMa icon indicating copy to clipboard operation
GPTQ-for-LLaMa copied to clipboard

Published 20 hours ago • verified publisher icon qwopqwop200

Porting GPTQ to CPU?

Open yiliu30 opened this issue 2 years ago • 2 comments

Is it possible to run GPTQ on a machine that has only CPUs? If not, is there a plan for it?

yiliu30 avatar May 22 '23 08:05 yiliu30

You can use a GPTQ quantized model with llama.cpp by using this conversion script I believe.

aljungberg avatar May 22 '23 14:05 aljungberg

just quant model on CPU?

Hiwyl avatar Nov 23 '23 13:11 Hiwyl