Bikkies
Bikkies
Have you followed this? https://github.com/oobabooga/text-generation-webui/blob/main/docs/GPTQ-models-(4-bit-mode).md#using-loras-in-4-bit-mode
We need a way to specify numbers outside of the slider range. Even for things like gpu-layers, where it can calculate them wrong and not allow you to offload all...
> 100.000 context need 12GB VRAM, if its cached in RAM it would be 10 times slower, you realy need that? Yes, for some tasks I need that. Models coming...