microsoft
Anthropic client should support 'thinking' argument
What happened?
To take full advantage of the latest Anthropic model, Claude 3.7 Sonnet, the Anthropic client should support the thinking parameter as documented here.
Maybe this should count as a feature request, but it's so trivial I wasn't sure it qualified.
I think literally all that needs to be done is to add 'thinking' to this line:
# Optional parameters
for param in ["top_p", "top_k", "stop_sequences", "metadata"]:
And probably add it as a constructor parameter as well.
I have another question, though. Currently, almost nothing seems to be using the extra_create_args parameter. What if I want different AssistantAgents to use different arguments (including temperature and others). It seems that if I want this, I need to construct different clients for each set of arguments. But that feels a little odd. Is it by design?
I'd like to go a step further and allow an AssistantAgent's parameters to change over its lifetime. I have agents which, depending on the circumstances, may want more predictable or more creative output. So I would like the ability to change the temperature and top-p dynamically. It would seem logical to me to do this by calling a method on the agent (possibly triggered by a tool, either invoked by the agent itself or a coordinating agent). Again, is there a design reason for not allowing this, or has the use case just not come up often enough?
I'm happy to submit pull requests to add the necessary functionality (both the thinking parameter and the other changes discussed above), I just wanted to check that I'm not going down the wrong path.
Which packages was the bug in?
Python AgentChat (autogen-agentchat>=0.4.0)
AutoGen library version.
Python dev (main branch)
Other library version.
No response
Model used
claude-3-7-sonnet-20250219
Model provider
Anthropic
Other model provider
No response
Python version
None
.NET version
None
Operating system
None
@shatteredLuigi ,
I think you can go ahead and submit a fix for the thinking param, thanks!
Other changes might need more discussion. Perhaps, one important design philosophy for AutoGen is that we follow the Actor Model. Part of the tenets here is that external entities should not mutate the state of an agent directly, but instead do this through message passing. This requirement enables more scalable designs.
What if I want different AssistantAgents to use different arguments (including temperature and others). It seems that if I want this, I need to construct different clients for each set of arguments. But that feels a little odd. Is it by design?
Were you thinking of sharing model clients across multiple AssistantAgents?
To the use case of agents with varied creativity .. what comes to mind is as follows
- Instantiate multiple agents with various params and route to them as needed (more code but easier mental model)
- Custom agent https://microsoft.github.io/autogen/dev/user-guide/agentchat-user-guide/custom-agents.html
I think you can go ahead and submit a fix for the thinking param, thanks!
Will do; I don't have a development environment set up for it yet but I assume you'd prefer I add to the .NET client as well?
Were you thinking of sharing model clients across multiple AssistantAgents?
Shows how much I know; I was assuming that was the standard approach. I've been using Google's genai SDK and using a single client for everything without issue, and the small amount of proof of concept code I wrote using AutoGen worked with a single client.
Instantiate multiple agents with various params and route to them as needed
Meaning I would have a custom agent that encapsulates multiple AssistantAgents, each with a different model client with different parameters, but which all share the same memory and context? And the custom agent would choose one of the child agents to handle each given message? I suppose that would work, but it seems needlessly complicated, and doesn't match my mental model of a single agent with creativity that can vary over time. It also wouldn't work so well if I want to support any arbitrary combination of parameters which can't be predicted in advance.
I could of course create my own custom agent based off of AssistantAgent, which I suppose would either have a built-in tool to change its own parameters or receive a custom message type from another agent to change its parameters. But I had sort of hoped to leverage AssistantAgent as-is and not fork large amounts of code, if it could be avoided.
Realized I forgot to @ you, @victordibia (no rush, just wanted to make sure you saw my response)
Meaning I would have a custom agent that encapsulates multiple AssistantAgents
No, you can create a single agent and enable varied creativity.
- implement its own model_cleint
- it can process the received message (or its own history of messages) and decide the parameters to use with a model client. All you have to do is modify the
on_messages_streammethod here.
A couple of line changes to the sample here would give you that behaviour.
@victordibia I'm probably overlooking something fundamental, but I'm still not seeing how it's that simple... The sample is inheriting from BaseChatAgent, but what I would like is an AssistantAgent—which has a memory, reflects on tool use, etc.—which also has dynamic temperature/top-p.
Even if I have a custom model client, something would need to pass the parameters into its create method, no? In AssistantAgent that happens in _call_llm, which is a private method, so it doesn't seem like there's a clean way for me to extend AssistantAgent to pass the additional parameters; I would have to fork it and modify _call_llm to use extra_create_args.
One other option would be for me to extend AssistantAgent, and override on_messages_stream to decide the parameters, pass them to the custom model client by calling some method (set_temperature), and then call super().on_messages_stream. The custom model client would override create, add the temperature from set_temperature to extra_create_args, and call super().create. Or I could do the same exact thing with composition instead of inheritance.
In both cases though, that feels like a "hack" which goes against the Actor Model you described earlier...
Anyway, sorry for getting so off-topic, and thanks for your time. Will submit a PR when I have some time this week or weekend.
@shatteredLuigi, for any type of instance-based update, would you consider an alternative of on-demand instance creation? What I meant is, can you structure your application so that it doesn't relies on instances like AssistantAgent to be constantly in-memory and modified, rather, you create instance on the fly upon user session with appropriate parameters like temperature?
This way your application always keeps a persisted state and is much more resilient to failures.
Agents are just wrappers, it's very cheap to create and destroy instances of agents.
@ekzhu I thought about that, but if the only way for an AssistantAgent to use different temperature/top-p is to use a different model client, and I want this to work with any arbitrary client, I would need to give my CustomAgent a ChatCompletionClientFactory to construct new instances, and implement a factory for each of the types of clients I want to support. Possible, yes, but it still feels like unnecessary complexity given how simple what I'm trying to do is.
At this point I'm thinking I'll probably just implement my own lightweight framework, since I plan on (and am currently) doing a lot of other funky things that are much more complex, and I feel like I would be trying to fit a square peg in a round hole trying to use AutoGen.
I still do appreciate its design though, and plan to fix this issue.
Got it. I think if you are experimenting with agent deign it's better to start from scratch. The Core API can help when you have multiple agents.
To clarify my point on creating instances on the fly -- you can create instances from dictionary, for example.
from autogen_core.models import ChatCompletionClient
model_client_config = {
"provider": "autogen_ext.models.openai.OpenAIChatCompletionClient",
"component_type": "model",
"version": 1,
"component_version": 1,
"config": {"model": "gpt-4o"},
}
client = ChatCompletionClient.load_component(model_client_config)
You can also do similar thing with agents and other components in AutoGen. https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/serialize-components.html