openai-api icon indicating copy to clipboard operation
openai-api copied to clipboard

Added support for reasoning models

Open suddenlyAstral opened this issue 9 months ago • 0 comments

Because reasoning models a la o1-mini don't return all generated tokens the variable max_tokens was renamed in the official API to max_completion_tokens. max_tokens is consifered deprecated for chat models but remains supported in older models and is the only supported option for (non-chat) completion API.

When using reasoning models set temperature: None, top_p: None. The API supports only those values (and temp=1.0 which is equivalent)

This PR:

  1. renames max_tokens -> max_completion_tokens in ChatBody to support reasoning models
  2. added reasoning model test
  3. deals with 0.1.9 non-compatibility with reasoning API in the narrowest way possible

This PR is not:

  1. Backward compatible. Updating to this version requires swapping max_tokens -> max_completion_tokens in every existing use
  2. providing a clear error message if you set temperature and top_p to an illegal value

suddenlyAstral avatar Feb 22 '25 15:02 suddenlyAstral