AlpinDale
AlpinDale
The quant name in aphrodite is unfortunately a bit misleading - I intend to fix this with the next release. The `load_in_4bit` quant isn't actually bitsandbytes, it's SmoothQuant+. We don't...
Might be a good idea to keep it open in case someone else has the same issue. I'll close it myself once we have real bitsandbytes support.
Bitsandbytes loading now supported.
It's definitely planned. I haven't looked at the architecture yet, but I assume it's RNN-based, so #419 would probably make ways for it. We have a lot more to work...
Please keep the issue open for tracking purposes.
Oh boy, good catch. I'll fix this ASAP.
Aphrodite filters out consolidated weights from mistral and .pth files from llama repos by default as of v0.6.0
Can you test this again with v0.6.0 please?
Good call, I didn't think of that. I'll make it so that the dtype flag takes priority over it for the next release.
We no longer force this as of v0.6.0