grok-1 icon indicating copy to clipboard operation
grok-1 copied to clipboard

Published 20 hours ago • verified publisher icon xai-org

4bit quantization

Open fakerybakery opened this issue 1 year ago • 3 comments

Hi, Thanks for releasing Grok! Is there any chance we could load the model in 4-bit given how large it is? Do you know if bitsandbytes support is planned (cc @timdettmers)? Thanks!

fakerybakery avatar Mar 17 '24 20:03 fakerybakery