mudakisa
Results
2
comments of
mudakisa
Well, for me (with 4090) 30b 4bit works, yes. But after context tokens reach 1000+, 24 GB VRAM seems not enough. I start to get problems (response with 0 tokens)....
> Thank you for the responses. That does my question. It's not a big problem if it's just a bit slower. I was just worried because the one from the...