llm
llm copied to clipboard
Support the bit-shuffling changes from `llama.cpp`
A new file version is being introduced to change how the tensors are stored on-disk: https://github.com/ggerganov/llama.cpp/pull/1305
We will need to support this version, as well as the older versions.
It's been merged: https://github.com/ggerganov/llama.cpp/pull/1405
There doesn't seem to be a migration path at present, so let's wait a bit: https://github.com/ggerganov/llama.cpp/issues/1408
This is done in #226, but I'd like to set up a migration path before I close this
No migration path for now. See #261