gemini-cli icon indicating copy to clipboard operation
gemini-cli copied to clipboard

feat(routing): make model router configurable

Open rmedranollamas opened this issue 2 months ago • 1 comments

This change introduces the ability to configure the models used by the model router through the settings.json file.

Users can now specify simpleTaskModel and complexTaskModel within the experimental.router object in their settings to override the default models used for simple and complex tasks.

This enhances the flexibility of the model router, allowing for the use of fine-tuned or company-specific models.

TLDR

This pull request introduces the ability to configure the models used by the experimental model router. Users can now specify a simpleTaskModel and a complexTaskModel in their settings.json file, allowing for greater flexibility and the use of fine-tuned or company-specific models.

Dive Deeper

The primary goal of this change is to decouple the model router's logic from hardcoded model names. Previously, the ClassifierStrategy would always choose between gemini-2.5-pro and gemini-2.5-flash. This PR modifies the configuration schema and the core Config object to read custom model names from the experimental.router section of a user's settings.

The changes include:

  • settingsSchema.ts: Updated to define the new experimental.router object with simpleTaskModel and complexTaskModel properties, including default values for backward compatibility.
  • config.ts (core): The Config class now stores these model names and provides getSimpleTaskModel() and getComplexTaskModel() getters.
  • config.ts (cli): The loadCliConfig function now reads the new settings and passes them to the Config object.
  • classifierStrategy.ts: The strategy now uses the new getters on the Config object to determine which model to use, instead of relying on hardcoded constants.
  • Tests: Added and updated tests for the settings schema, Config object, and ClassifierStrategy to ensure the new functionality is working as expected and to prevent regressions.

Reviewer Test Plan

To validate these changes, you can perform the following steps:

  1. Create a settings.json file in the root of the project with the following content:

    {
      "experimental": {
        "useModelRouter": true,
        "router": {
          "simpleTaskModel": "my-custom-flash",
          "complexTaskModel": "my-custom-pro"
        }
      }
    }
    
  2. Run the CLI with debug mode enabled to see the internal logging from the model router.

    For a simple task:

    DEBUG_MODE=true npm start -- "list the files in the current directory"
    

    Look for a debug log indicating that my-custom-flash was selected.

    For a complex task:

    DEBUG_MODE=true npm start -- "how would I add a new command to this CLI?"
    

    Look for a debug log indicating that my-custom-pro was selected.

  3. Verify Default Behavior: Remove the settings.json file and run the same commands. The router should fall back to the default models (gemini-2.5-flash and gemini-2.5-pro).

Testing Matrix

🍏 🪟 🐧
npm run
npx
Docker
Podman - -
Seatbelt - -

Linked issues / bugs

This PR does not resolve any specific issues.

rmedranollamas avatar Oct 16 '25 20:10 rmedranollamas

For the settings, should they be factored to look something like:

{
  "experimental": {
    "modelRouter": {
      "enable": true,
      "simpleTaskModel": "my-custom-flash",
      "complexTaskModel": "my-custom-pro"
    }
  }
}

So that there is one top level settings obj for the model router?

ptone avatar Oct 16 '25 20:10 ptone