NiLon

Min value doesn't have effect on empty strings.

Isn't mlock just reserving (locking) the memory for the model for the maximum amount rather than dynamically increasing as the model needs it? I don't think it actually contributes towards...

> Well I think it would but Indeed it would do more and force it constantly into memory, I might be wrong tho and it may not change this issue...

> I run it on cpu, the screenshot is taken using mixtral (while prompting). When using mistral (lighter model) it seems to load a part (still not all) of it...