langchain icon indicating copy to clipboard operation
langchain copied to clipboard

Issue: Model and model_name inconsistency in OpenAI LLMs such as ChatOpenAI

Open tabacof opened this issue 1 year ago • 0 comments

Issue you'd like to raise.

Argument model_name is the standard way of defining a model in LangChain's ChatOpenAI. However, OpenAI uses model in their own API. To handle this discrepancy, LangChain transforms model_name into model here.

The problem is that, if you ignore model_name and use model in the LLM instantiation e.g. ChatOpenAI(model=...), it still works! It works because model becomes part of model_kwargs, which takes precedence over the default model_name (which would be "gpt-3.5-turbo"). This leads to an inconsistency: the model can be anything (e.g. "gpt-4-0314"), but model_name will be the default value.

This inconsistency won't cause any direct issue but can be problematic when you're trying to understand what models are actually being called and used. I'm raising this issue because I lost a couple of hours myself trying to understand what was happening.

Suggestion:

There are three ways to solve it:

  1. Raise an error or warning if model is used as an argument and suggest using model_name instead
  2. Raise a warning if model is defined differently from model_name
  3. Change from model_name to model to make it consistent with OpenAI's API

I think (3) is unfeasible due to the breaking change, but raising a warning seems low effort and safe enough.

tabacof avatar May 08 '23 10:05 tabacof