zed icon indicating copy to clipboard operation
zed copied to clipboard

Add max_output_tokens to OpenAI models and integrate into requests

Open Cupnfish opened this issue 1 year ago • 4 comments

Pull Request Title

Introduce max_output_tokens Field for OpenAI Models

https://platform.deepseek.com/api-docs/news/news0725/#4-8k-max_tokens-betarelease-longer-possibilities

Description

This commit introduces a new field max_output_tokens to the OpenAI models, which allows specifying the maximum number of tokens that can be generated in the output. This field is now integrated into the request handling across multiple crates, ensuring that the output token limit is respected during language model completions.

Changes include:

  • Adding max_output_tokens to the Custom variant of the open_ai::Model enum.
  • Updating the into_open_ai method in LanguageModelRequest to accept and use max_output_tokens.
  • Modifying the OpenAiLanguageModel and CloudLanguageModel implementations to pass max_output_tokens when converting requests.
  • Ensuring that the max_output_tokens field is correctly serialized and deserialized in relevant structures.

This enhancement provides more control over the output length of OpenAI model responses, improving the flexibility and accuracy of language model interactions.

Changes

  • Added max_output_tokens to the Custom variant of the open_ai::Model enum.
  • Updated the into_open_ai method in LanguageModelRequest to accept and use max_output_tokens.
  • Modified the OpenAiLanguageModel and CloudLanguageModel implementations to pass max_output_tokens when converting requests.
  • Ensured that the max_output_tokens field is correctly serialized and deserialized in relevant structures.

Related Issue

https://github.com/zed-industries/zed/pull/16358

Release Notes

  • Added max_output_tokens field to OpenAI models for controlling output token length.
  • Improved request handling across multiple crates to respect max_output_tokens.

Screenshots / Media

N/A

Checklist

  • [x] Code compiles correctly.
  • [x] All tests pass.
  • [ ] Documentation has been updated accordingly.
  • [ ] Additional tests have been added to cover new functionality.
  • [ ] Relevant documentation has been updated or added.

Cupnfish avatar Aug 16 '24 20:08 Cupnfish

We require contributors to sign our Contributor License Agreement, and we don't have @Cupnfish on file. You can sign our CLA at https://zed.dev/cla. Once you've signed, post a comment here that says '@cla-bot check'.

cla-bot[bot] avatar Aug 16 '24 20:08 cla-bot[bot]

@cla-bot check

Cupnfish avatar Aug 16 '24 20:08 Cupnfish

The cla-bot has been summoned, and re-checked this pull request!

cla-bot[bot] avatar Aug 16 '24 20:08 cla-bot[bot]

Warnings
:warning:

This PR is missing release notes.

Please add a "Release Notes" section that describes the change:

Release Notes:

- Added/Fixed/Improved ...

If your change is not user-facing, you can use "N/A" for the entry:

Release Notes:

- N/A

Generated by :no_entry_sign: dangerJS against d0c00cd2fc6ed9c6cd2f5760c440fac731af31ba

zed-industries-bot avatar Aug 16 '24 20:08 zed-industries-bot

FYI. This broke GPT-4 (and only GPT-4). image Fix in:

  • https://github.com/zed-industries/zed/pull/17168

notpeter avatar Aug 30 '24 17:08 notpeter