💡 [REQUEST] - Please consider support Mac chips compatible quantization method

[thiner]

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

相关Issues | Reference Issues

No response

摘要 | Summary

AutoGPTQ doesn't support MacOS till now. Could you consider support Mac compatible quantization method? E.g. gguf.

基本示例 | Basic Example

I am taking llama-cpp-python as an example: https://llama-cpp-python.readthedocs.io/en/latest/install/macos/

缺陷 | Drawbacks

n/a

未解决问题 | Unresolved questions

No response

[qizhanchan]

Need gguf +1 to load it for ollama