gguf-tools
gguf-tools copied to clipboard
Add IQ2 tensor types
These were added in https://github.com/ggerganov/llama.cpp/pull/4773
It's annoying that I8 used to be 16 and it's now 18. I16 and I32 also changed.
Dequantization code is very cryptic. I would love to see your take :)