hollama
hollama copied to clipboard
Configure default parameters for models
Also stuff like "use mmap" or thread count Originally posted by @binarynoise in #279
There's many ways we can go about this:
- Apply model settings globally. This will mean that all of the models from any server will default to the user-defined parameters. This is how OpenWebUI handles it AFAIK.
- Apply the settings per server/connection. Means that the models from that server will default to the user-defined parameters.
- Apply the settings per model. User-defined settings will only be applied when a particular model is used, all others will use the provider's (Ollama/OpenAI) defaults.
Without having done supe deep research, option 1 is probably the easiest implementation of such a feature.
@binarynoise do you have any insights on how this would play out in your workflow?
I would prefer option 1, as this would allow using default settings also when using a private tab, my laptop/phone, or when a friend uses my hollama from their PC.
Option 2 would be okayish, because we would lose the benefit of defaults for new clients but would need only one setup per client.
I don't like option 3 because when switching models around a lot, I'd need to apply the same settings over and over. I would only add this as an additional layer.