MillionthOdin16
MillionthOdin16
Awesome! Worked for me too. I completely forgot to rebuild the kernel -_-
Yes, that's the issue. It needs to be the cuda ones with no act order.
I'm also getting this. 64GB system ram and 24 VRAM. The model is like 18GB, and I have more than enough ram to handle it on both types.
If you're familiar with the mixing of loras I think it would be helpful for a lot of people here if you could link some resources on it. I've heard...
Is this changing the default number of threads from the intended 4 to 8? I can't tell from a quick read, but if that's the case, eight seems a bit...
> @Zetaphor Correct, llama.cpp has set the default token context window at 512 for performance, which is also the default `n_ctx` value in [langchain](https://github.com/hwchase17/langchain/blob/master/langchain/llms/llamacpp.py#L30). You can set it at 2048...
No, it's broken. It works on hugging face now but can't download loras xD.
To be honest I'm trying to help you, but the effort put into this issue to explain what's going on is so low... You took the time to dump the...
Pretty sure the default version of the code uses like 4. Or at least the initial examples.
Oh damn. That's why people are complaining when they use all their threads 😂 I guess that's the one bonus Windows has in this case. On Thu, Apr 13, 2023,...