ggml icon indicating copy to clipboard operation
ggml copied to clipboard

Published 20 hours ago • verified publisher icon ggerganov

[Feature request] Implement 8-bit GPT-J

Open pablogranolabar opened this issue 2 years ago • 0 comments

Results in ~11Gb weights vs. 16Gb, implemented in PyTorch now as load_in_8bit=True:

https://huggingface.co/hivemind/gpt-j-6B-8bit

pablogranolabar avatar Nov 13 '22 18:11 pablogranolabar