anything-llm icon indicating copy to clipboard operation
anything-llm copied to clipboard

Published 20 hours ago • verified publisher icon Mintplex-Labs

Invalid OpenAI Token Limit

Open imbible opened this issue 1 year ago • 1 comments

The Azure OpenAI has the GPT-4 turbo model, which has 128,000 tokens. I am able to select that from the dropdown list, but cannot save the settings. Here is the option: image Here is the error message: image This is a screenshot from my Azure OpenAI Chat playground. As you can see, I do have the deployment with 128,000 tokens. image

I believe this line is wrong. https://github.com/Mintplex-Labs/anything-llm/blob/6d5968bf7e694aae8e68bbb4cd2e0e6f491f47d1/server/utils/helpers/updateENV.js#L296 I'm not sure if this is the root cause.

imbible avatar Dec 31 '23 18:12 imbible

Ah, the token count for this model is indeed missing. Good catch. Will mark as bug

timothycarambat avatar Dec 31 '23 19:12 timothycarambat