AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard
verified publisher icon casper-hansen

Add Cohere Support

Open localbarrage opened this issue 1 year ago • 1 comments

https://huggingface.co/CohereForAI/c4ai-command-r-v01

The new Cohere model is #1 trending on huggingface right now. It excels at RAG, Tool Usage (Json generation), etc. It is a 35B parameter model so AWQ quantization support would be nice.

localbarrage avatar Mar 21 '24 02:03 localbarrage

This model seems incredibly useful. Since this is a dense model, adding quantization support should be a bit easier. I will experiment soon with this model to see if we can quantize it.

casper-hansen avatar Apr 06 '24 10:04 casper-hansen