Feature request: Add support for Claude 3.7 Sonnet thinking mode configuration in BedrockLLMAgent

[astoloff19]

Use case

Claude 3.7 Sonnet introduces a dual-mode capability that combines standard LLM responses with an enhanced reasoning mode. In extended thinking mode, the model performs self-reflection before answering, which significantly improves its performance on complex tasks.

For the Multi-Agent Orchestrator, we want to enable this reasoning capability specifically for supervisor agents to better break down and coordinate complex tasks across multiple specialized agents. Currently, the BedrockLLMAgent class doesn't support configuring this feature through its options.

Solution/User Experience

1. Extend BedrockLLMAgentOptions to include thinking mode configuration:

@dataclass
class BedrockLLMAgentOptions(AgentOptions):
    # ... existing fields ...
    thinking_config: Optional[dict[str, Any]] = None  # New field for thinking mode

2. Update the process_request method to include thinking configuration in the converse command:

async def process_request(self, ...):
    converse_cmd = {
        'modelId': self.model_id,
        'messages': conversation_to_dict(conversation),
        'system': [{'text': system_prompt}],
        'inferenceConfig': {
            # ... existing inference config ...
        }
    }
    
    # Add thinking configuration if specified
    if self.thinking_config:
        converse_cmd["additionalModelRequestFields"] = {
            "thinking": self.thinking_config
        }

3. Example usage:

agent = BedrockLLMAgent(BedrockLLMAgentOptions(
    name="ReasoningAgent",
    description="An agent that shows its reasoning",
    thinking_config={
        "type": "enabled",
        "budget_tokens": 2000
    }
))

4. Add support in handle_single_response and handle_streaming_response to process and return thinking content when available.

This enhancement would allow developers to optionally enable and configure the thinking mode when needed, while maintaining backward compatibility with existing implementations.

Alternative solutions

[raghavgrover13]

This is indeed a very important feature to add especially with Agentic AI systems, we are building production grade systems with this framework and if this feature can be prioritized it would be of great help.

[rmadabusi-info]

This is indeed a very important feature to add especially with Agentic AI systems, we are building production grade systems with this framework and if this feature can be prioritized it would be of great help.

Hello @raghavgrover13 , I'm very curious to know more about your statement of using this framework for production grade systems. In my quick test, I stumbled on an issue - https://github.com/awslabs/multi-agent-orchestrator/issues/265 that uses sync function for an async call and wonder how your test never hit this issue. Is it some race or rare condition that i hit with? Given the attention of original coders on the issues reported of this framework, I'm bit uncomfortable in using this for production grade but would love to hear your opinion.

[raghavgrover13]

This is indeed a very important feature to add especially with Agentic AI systems, we are building production grade systems with this framework and if this feature can be prioritized it would be of great help.

Hello @raghavgrover13 , I'm very curious to know more about your statement of using this framework for production grade systems. In my quick test, I stumbled on an issue - #265 that uses sync function for an async call and wonder how your test never hit this issue. Is it some race or rare condition that i hit with? Given the attention of original coders on the issues reported of this framework, I'm bit uncomfortable in using this for production grade but would love to hear your opinion.

We didn't face this issue and we have 3 agentic chat systems live, so could be some rare condition, but anyways this issue seems to be reported fixed in 0.1.10

[raghavgrover13]

If anyone can work on this, it would be great. Supervisor or agent having thinking mode with 3.7 sonnet wud be amazing for some tough scenarios.

[brnaba-aws]

Thanks for raising this, we will start working on it.