AzureOpenAI can't support other model like text-davinci-003

[jacobjiangwei]

System Info

llm = AzureChatOpenAI(deployment_name="JacobTextDavinci03", model_name="text-davinci-003", max_tokens=1500)

File "C:\Users\jwei\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\openai\api_requestor.py", line 687, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: Resource not found

It works well if I use gpt-3.5-turbo model. So I'm guessing it's something wrong with openai library that can't support text-davinci-003 ?

Who can help?

@hwchase17

Information

• [X] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

It's quite easy to reproduce. Just use follow example here : https://python.langchain.com/en/latest/modules/models/llms/integrations/azure_openai_example.html

And it just won't work for other model except 3.5 gpt turbo.

Expected behavior

it should support azure all kinds of model there.

[zioproto]

Only the gpt-3.5-turbo and gpt-4 models are chat models and support using the Chat API via llm = AzureChatOpenAI(......).

The other models are completion models, you should use the Completion API with llm = AzureOpenAI(....)

Review the python examples here:

• https://learn.microsoft.com/en-us/azure/cognitive-services/openai/quickstart?pivots=programming-language-python&tabs=command-line
• https://learn.microsoft.com/en-us/azure/cognitive-services/openai/chatgpt-quickstart?pivots=programming-language-python&tabs=command-line

Please check closely the difference between openai.Completion.create and openai.ChatCompletion.create in the 2 documentation pages

Please let me know if you have questions.

[vincentteyssier]

I use gpt-35-turbo and I still have the issue

[zioproto]

@vincentteyssier if you get the error openai.error.InvalidRequestError: Resource not found it means that the API call is returning a HTTP 404.

Please double check the model_name and deployment_name and the api_base if you configured a custom DNS.

You want to make sure you are making the HTTP request to the correct API endpoint.

[vincentteyssier]

Thanks, I found out what the issue is. On Azure Portal, the API version mentioned is 0301. With that settings it does not work. If I use '2023-03-15-preview' instead, then it works. Funny I get this issue only with AzureChatOpenAI, but not with AzureOpenAI

[zioproto]

The number 0301 is the version of the model, not the Azure API version. The Azure API version is 2023-03-15-preview

[zioproto]

Funny I get this issue only with AzureChatOpenAI, but not with AzureOpenAI

It is because only AzureChatOpenAI requires openai_api_version. https://github.com/hwchase17/langchain/blob/5cfa72a130f675c8da5963a11d416f553f692e72/langchain/chat_models/azure_openai.py#L46

Look at AzureOpenAI, it wants only the deployment_name: https://github.com/hwchase17/langchain/blob/5cfa72a130f675c8da5963a11d416f553f692e72/langchain/llms/openai.py#L571-L588

This is because AzureOpenAI extends the class BaseOpenAI: https://github.com/hwchase17/langchain/blob/5cfa72a130f675c8da5963a11d416f553f692e72/langchain/llms/openai.py#L123 that is never setting the API version when using the OpenAI python module:

https://github.com/hwchase17/langchain/blob/5cfa72a130f675c8da5963a11d416f553f692e72/langchain/llms/openai.py#L227-L235

so when you use AzureOpenAI you are reading the API version from the environment variable OPENAI_API_VERSION

[dosubot[bot]]

Hi, @jacobjiangwei! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you reported was about the "text-davinci-003" model in AzureOpenAI not being supported by the openai library, resulting in a "Resource not found" error. There have been some helpful discussions in the comments. @hwchase17 was sought for help, and @zioproto explained that only the "gpt-3.5-turbo" and "gpt-4" models are chat models and support the Chat API, while other models should use the Completion API.

However, @vincentteyssier discovered that using the API version "2023-03-15-preview" instead of "0301" resolves the issue. @zioproto also explained that the difference in behavior between AzureChatOpenAI and AzureOpenAI is due to the former requiring the "openai_api_version" while the latter reads it from the environment variable.

Based on this information, it seems that the issue has been resolved by using the API version "2023-03-15-preview" instead of "0301".

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your contribution to the LangChain repository!