Adds Supports for OpenAI Reasoning Models

[gagb]

This pull request introduces support for a new set of models (o1 and o1-mini) in the OpenAI client.

Task List

• [x] Create new api_type
• [x] Add costs/token limits
• [ ] Support max_completion_tokens
• [ ] Display warning for multi-model messages?

New Client Support:

• autogen/oai/client.py: Added a new class OpenAI_O1 to handle the o1 models, including a method to modify the role of messages before processing. Updated the client registration logic to include the new OpenAI_O1 client. [1] [2]

Model Configuration:

• autogen/oai/openai_utils.py: Added configuration for o1 and o1-mini models, including their respective costs.
• autogen/token_count_utils.py: Updated the token limit configurations to include o1 and o1-mini models.

Testing:

• test/twoagent-o1.py: Added a new sample file to show the integration of o1 models using AssistantAgent and UserProxyAgent.

Related: #3523

PS: I am going out for the evening; the new models feel slower for sure but return more detail responses. I bet the "system" prompts will need some tuning even though they aren't supported!

[marklysze]

Would love to test but don't have tier 5 API access :) can look through code.

Note the current Beta Limitations:

• Modalities: text only, images are not supported.
• Message types: user and assistant messages only, system messages are not supported. [DONE]
• Streaming: not supported.
• Tools: tools, function calling, and response format parameters are not supported.

A few questions:

• I can see that it's easier to handle the beta limitations through a new class/api_type, OpenAI_O1, though do you think these models will always warrant a different one?
• Do you want to strip out streaming from params?
• If max_tokens is in the params, it could be changed to max_completion_tokens (it looks like this is a common change to the OpenAI API, so would apply to all models)
• Should system messages be updated with assistant or user? I thought the user may be more appropriate as an instruction to the model?

[gagb]

Would love to test but don't have tier 5 API access :) can look through code.

That's okay I really appreciate the feedback on the code :)

Note the current Beta Limitations:

• Modalities: text only, images are not supported.
• Message types: user and assistant messages only, system messages are not supported. [DONE]
• Streaming: not supported.
• Tools: tools, function calling, and response format parameters are not supported.

A few questions:

• I can see that it's easier to handle the beta limitations through a new class/api_type, OpenAI_O1, though do you think these models will always warrant a different one?

Good question. Hard to know what they will do. But I am open to different solutions. The reason I like this is because it modular and backwards compatible. If they make further upgrades, we can depreciate this client?

• Do you want to strip out streaming from params?
• If max_tokens is in the params, it could be changed to max_completion_tokens (it looks like this is a common change to the OpenAI API, so would apply to all models)

Good point, I was trying to first get the basic version working. Will add this.

• Should system messages be updated with assistant or user? I thought the user may be more appropriate as an instruction to the model?

You are correct. This was a typo.

[marklysze]

• I can see that it's easier to handle the beta limitations through a new class/api_type, OpenAI_O1, though do you think these models will always warrant a different one?

Good question. Hard to know what they will do. But I am open to different solutions. The reason I like this is because it modular and backwards compatible. If they make further upgrades, we can depreciate this client?

I think that works :)

• If max_tokens is in the params, it could be changed to max_completion_tokens (it looks like this is a common change to the OpenAI API, so would apply to all models)

Good point, I was trying to first get the basic version working. Will add this.

Thanks!

• Should system messages be updated with assistant or user? I thought the user may be more appropriate as an instruction to the model?

You are correct. This was a typo.

Nice, all good :)

[rysweet]

looks like some open questions still... @gagb

[yeandy]

This PR is only enabling the o1 model for AutoGen 0.2, right?

The counterpart for AutoGen 0.4 is this one https://github.com/microsoft/autogen/issues/3884, right? (which I imagine is a bit more complicated to implement?)

[ekzhu]

This PR is only enabling the o1 model for AutoGen 0.2, right?

The counterpart for AutoGen 0.4 is this one #3884, right? (which I imagine is a bit more complicated to implement?)

I believe for v0.4 support it should be as simple as allowing no system message in AssistantAgent.