SayanoAI
idea: add toppy, use min p
Toppy's a great 7B model - the best, in my opinion, and many others have also loved it for RP. Aside from that, Min P is a sampler implemented in every open source LLM backend. (except transformers)
You can install the model yourself by putting it in the models/LLM folder and creating your own model config. Last time I checked, Min P hasn't been implemented to the main koboldcpp release package yet. Let me know if that has changed.
You can install the model yourself by putting it in the models/LLM folder and creating your own model config. Last time I checked, Min P hasn't been implemented to the main koboldcpp release package yet. Let me know if that has changed.
..when did you last check? they added min p on november 4
also, i meant adding a config for toppy, as its the most used and most praised 7B iirc based on mistral for rp