LaaZa

I'm not sure what the problem is, might be your model path, you should use huggingface id instead of manually using a path from the cache. It won't redownload it...

Isn't that a mistake on @TheBloke s part? I don't think it makes sense for these values to be lists in the [quantize_config.json](https://huggingface.co/TheBloke/stablelm-zephyr-3b-GPTQ/blob/main/quantize_config.json)

AutoGPTQ is made fo CausalLM, not vision models. Also 8-bit is probably not worth it with GPTQ, you could try bitsandbytes.