llama.cpp qwen model quantized with AWQ and lora weights

qwen model quantized with AWQ and lora weights

Open shuyuan-wang opened this issue 1 month ago • 0 comments

Hello, I'm very new to this repo and just read through the quickstart. I am curious does this repo support qwen model quantized with AWQ which has int4 weights and lora weights trained upon the quantized weight (fp16 I think)?

Jan 17 '25 09:01 shuyuan-wang

llama.cpp llama.cpp copied to clipboard

qwen model quantized with AWQ and lora weights

llama.cpp
llama.cpp copied to clipboard