feat(routing): make model router configurable
This change introduces the ability to configure the models used by the model router through the settings.json file.
Users can now specify simpleTaskModel and complexTaskModel within the experimental.router object in their settings to override the default models used for simple and complex tasks.
This enhances the flexibility of the model router, allowing for the use of fine-tuned or company-specific models.
TLDR
This pull request introduces the ability to configure the models used by the experimental model router. Users can now specify a simpleTaskModel and a complexTaskModel in their settings.json file, allowing for greater flexibility and the use of fine-tuned or company-specific models.
Dive Deeper
The primary goal of this change is to decouple the model router's logic from hardcoded model names. Previously, the ClassifierStrategy would always choose between gemini-2.5-pro and gemini-2.5-flash. This PR modifies the configuration schema and the core Config object to read custom model names from the experimental.router section of a user's settings.
The changes include:
settingsSchema.ts: Updated to define the newexperimental.routerobject withsimpleTaskModelandcomplexTaskModelproperties, including default values for backward compatibility.config.ts(core): TheConfigclass now stores these model names and providesgetSimpleTaskModel()andgetComplexTaskModel()getters.config.ts(cli): TheloadCliConfigfunction now reads the new settings and passes them to theConfigobject.classifierStrategy.ts: The strategy now uses the new getters on theConfigobject to determine which model to use, instead of relying on hardcoded constants.- Tests: Added and updated tests for the settings schema,
Configobject, andClassifierStrategyto ensure the new functionality is working as expected and to prevent regressions.
Reviewer Test Plan
To validate these changes, you can perform the following steps:
-
Create a
settings.jsonfile in the root of the project with the following content:{ "experimental": { "useModelRouter": true, "router": { "simpleTaskModel": "my-custom-flash", "complexTaskModel": "my-custom-pro" } } } -
Run the CLI with debug mode enabled to see the internal logging from the model router.
For a simple task:
DEBUG_MODE=true npm start -- "list the files in the current directory"Look for a debug log indicating that
my-custom-flashwas selected.For a complex task:
DEBUG_MODE=true npm start -- "how would I add a new command to this CLI?"Look for a debug log indicating that
my-custom-prowas selected. -
Verify Default Behavior: Remove the
settings.jsonfile and run the same commands. The router should fall back to the default models (gemini-2.5-flashandgemini-2.5-pro).
Testing Matrix
| 🍏 | 🪟 | 🐧 | |
|---|---|---|---|
| npm run | ✅ | ❓ | ❓ |
| npx | ❓ | ❓ | ❓ |
| Docker | ❓ | ❓ | ❓ |
| Podman | ❓ | - | - |
| Seatbelt | ❓ | - | - |
Linked issues / bugs
This PR does not resolve any specific issues.
For the settings, should they be factored to look something like:
{
"experimental": {
"modelRouter": {
"enable": true,
"simpleTaskModel": "my-custom-flash",
"complexTaskModel": "my-custom-pro"
}
}
}
So that there is one top level settings obj for the model router?