Hybrid Thinking Models

[i13-ctflora]

Is your feature request related to a problem? Please describe. There's a new hybrid thinking model, IBM Granite 3.2, that can run in either 'standard' mode, or in 'thinking' mode. This is similar to the approach used by Anthropic's Sonnet 3.7, which runs in 'standard' mode unless the 'thinking' mode is turned on in the JSON prompt object.

Describe the solution you'd like I'd like to be able to run this model in 'thinking' mode.

Describe alternatives you've considered The model currently runs fine in 'standard' mode, but I can't see any way to enable 'thinking' mode through the GUI or the copilot json.

Additional context I don't know if this will require an update from langchain, but here's what I've found:

According to Granite 3.2 model's implementation on Ollama, there's an additional line to add to the json message object: the line needs a "role" of "control". and a "content" of "thinking".

From the docs at https://ollama.com/library/granite3.2

{
    ...
    "messages": [
        {"role": "control", "content": "thinking"},
        {"role": "user", "content": "How do I get to the airport if my car won't start?"}
    ]
}

This is a bit different from enabling 'thinking' mode in Sonnet 3.7, where there's a separate "thinking" object at the same level as the "messages" object:

{
    ...
    "thinking": {
        "type": "enabled",
        "budget_tokens": 32000
    },
    "messages": [
        {
            "role": "user",
            "content": "Generate a comprehensive analysis of..."
        }
    ]
}