AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard

Published 20 hours ago • verified publisher icon casper-hansen

Add DBRX Support

Open Xu-Chen opened this issue 1 year ago • 2 comments

https://huggingface.co/databricks/dbrx-instruct DBRX Instruct is a mixture-of-experts (MoE) large language model trained from scratch by Databricks. DBRX Instruct specializes in few-turn interactions.

Xu-Chen avatar Mar 28 '24 06:03 Xu-Chen

Yes please

TypeError: dbrx isn't supported yet.

maziyarpanahi avatar Mar 28 '24 11:03 maziyarpanahi

I would love to add DBRX support. However, at the moment, I lack the hardware/$ to experiment enough to implement quantization support for this model because of the sheer size of it.

casper-hansen avatar Apr 06 '24 10:04 casper-hansen