simonw
Error messages and documentation improvement for incorrect model options
When I use an incorrect option for a particular model one get a very cryptic error message “Extra inputs are not permitted”:
llm 'Hello World!' -m qwen3 -o max_token 1000
Error: max_token
Extra inputs are not permitted
While the correct option “num_predict” of course works:
llm 'Hello World!' -m qwen3 -o num_predict 1000
Hello! 😊 How can I assist you today?
Besides improving the error message, it would also be good to have an example in the documentation on how one finds the available parameters. What I have found is to use:
llm models --options -m qwen3
But it took a long time to find this command, and there might be a better way! So the documentation around model options can be improved!
Also note that
llm -f docs: "How do I increase max output token?"
For all tested llm- models outputs something like “...specify the max_tokens option when running the command...”, which is true for OpenAI models, but not true for Ollama (‘num_predict’) or Gemini models (‘max_output_tokens’)! Thus my confusion when I saw the error message. Making note of these differences in the documentation would be very helpful!
I double the above comment. I have found this issue by using templates.
system: "Do this and that etc."
options:
temperature: 1.3
Since this template has no default model, running
llm -m 'my_model' -t 'my_template' 'My prompt...'
should work. And it does! Except only for the models that accept the 'temperature' parameter.
When I tested with gpt4all Llama-3 model, it failed with "Extra inputs are not permitted", because 'Llama-3' uses 'temp' instead of 'temperature'.
Checked: modifying the template to 'temp' works, though fails on some other models.
Ie. llm -m 'gemini-2.5-flash' -o temperature 1.3 'Hello.' works, yet using 'temp' does not.
This will always happen when a parameter is not accepted. I'd expect (wish - my opinion) that bad parameters would be filtered out before the inference or, better yet, adapted for each model's proper nomenclature.
With llm --options I can find details on model parameters, and then I can adjust my command, but, for the time being, the use case of a model-less template is undoable.
This issue somewhat crosses with https://github.com/simonw/llm/issues/724
May I suggest that a normalise_model_options function is developed that can translate between options?
It would sit after all options have been collected (after command line and template parsing) and before the model is actually called.
This function would assume three sets or lists: input_options {a dictionary of parameter options (strings) in pair with values (any basic type)}; model_options {a dictionary of parameters (strings) a model can accept with type and explanation etc.}; option_aliases {a list of set with option aliases (each element is a string)} maybe read from a file somewhere.
Then the function (e.g. called normalise_model_options) take these option sets and lists, together with a model name (for error purposes) and generates a valid output dictionary output_options.
The function should take the parameters in the input_option dict and check if it is in the allowed model_options, if not it should check if the parameter is in the alias set that then can be translated into an allowed model_option. All valid model options should be put in an output_option dict with the corresponding value. If a parameter is found in the alias set, and is not a valid model option, then an error should be returned with an error string like “The {parameter) is not a valid option for model {model_name}”
Example
# ── user‑supplied options ─────────────────────────────────────
input_options = { "max_token": 1000, "think": True,}
# ── alias groups (each group is a set of interchangeable names) ──
option_aliases = [
{"max_token", "num_predict", "max_output_tokens"},
{"temp", "temperature"},
]
# ── what the model actually understands (canonical names) ────────
model_options = {
"temperature": (float,
"The temperature of the model. Increasing the temperature will make the model answer more creatively. (Default: 0.8)"),
"num_predict": (int,
"Maximum number of tokens to predict when generating text. (Default: -1, infinite generation)"),
"think": (bool,
"Enable the model's thinking process."),
}
model_name = "qwen3"
Example output
output_options = {'num_predict': 1000, 'think': True}
This is an llm suggested header for this function:
def normalise_model_options(
*,
input_options: Mapping[str, Any],
model_options: Mapping[str, tuple[type, str]],
option_aliases: Sequence[Set[str]],
model_name: str,
) -> dict[str, Any]:
"""
Translate raw user options into a dict that only contains the *canonical*
option names accepted by the model.
1. Direct hit → keep.
2. Otherwise look for an alias set that also contains a canonical name.
Translate to that name and keep.
3. If no canonical name can be found → raise
InvalidOptionError(f"The option '{key}' is not a valid option for model {model_name}").
Returns a new dict whose keys are only canonical names.
"""
I could ask an llm to generate this, but I assume you want to do this within your own work flow.
I have implemented the functionality for this options adapter, however, some considerations I'd like to know your opinions about:
- Not sure about keeping the
click.echoes, since they interferes into the CLI model's output. - Maybe implement an option as
--adapt. Or leave it as default and implement--no-adapt? - Since this disregards any faulty/nonexistent keys, the former option validation code block turns irrelevant.
- What other common option aliases should be considered beyond the mentioned ones?
- I did not yet follow the contributing guides: am I supposed to publish a release of my own before calling a pull request?
Diff preview: https://github.com/fde-capu/llm/compare/main...options-adapting