ChatOpenAI() isn't a BaseLLM and therefore cannot use in ConversationalRetrievalQAChain

[clbarrell]

I'm trying to use the ConversationalRetrievalQAChain with GPT3.5 because it's cheaper than GPT3.

But when I try to use new ChatOpenAI() as the model for ConversationalRetrievalQAChain it gives an error because it doesn't have the properties of name and _generateUncached that are present in BaseLLM.

Is there anything I can do to resolve this for now?

const chatModel = new ChatOpenAI();

const chain = ConversationalRetrievalQAChain.fromLLM(
  chatModel, // << TYPE ERROR
  vectorStore.asRetriever(),
  { returnSourceDocuments: true }
);

[nsantos16]

I suggest use

const chatModel = new OpenAI({
      model: CHAT_GPT_MODEL
    });

[Achalogy]

class FixedBaseLLM extends BaseLLM {
  _generate(
    prompts: string[],
    stop?: string[] | undefined
  ): Promise<any> {
    throw new Error("Method not implemented.");
  }
  _llmType(): string {
    throw new Error("Method not implemented.");
  }
}

const llm: FixedBaseLLM = new OpenAIChat({
  openAIApiKey: ...,
});

const chain = ConversationalRetrievalQAChain.fromLLM(
  llm,
  this.Documents.asRetriever()
);

[ShantanuNair]

Wow, I didn't realize that openAI() was using GPT 3 instead of 3.5. Please, please make this more obvious in docs as it's not intuitive that the older, 10x more expensive model is the default. Unless for legacy reasons it stayed the same, but I'd imagine things are already moving too fast to be concerned with legacy right now.

@clbarrell @Achalogy @nsantos16 Have you got this working better than if you just called OpenAI() with modelName gpt-3.5-turbo ??

[Achalogy]

I think it won't work because completion and chat completion have different api endpoints.

[ShantanuNair]

@Achalogy looks like the constructor just calls chatOpenAI if the model name is gpt-3.5 turbo or gpt 4

[nelsonjchen]

https://github.com/hwchase17/langchainjs/blob/440500d81516ef9260e5da7a0c09a1d3a936ff4c/langchain/src/llms/openai.ts#L108

Done here.

Anyway, swapping this makes things just work indeed!

[dosubot[bot]]

Hi, @clbarrell! I'm helping the LangChain team manage their backlog and I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you raised was about the ChatOpenAI() model not being usable as the model for ConversationalRetrievalQAChain due to missing properties. There were some suggestions made, including using const chatModel = new OpenAI({ model: CHAT_GPT_MODEL }); as a possible solution. Additionally, there was a discussion about the default model being GPT 3 instead of 3.5 and the compatibility of different API endpoints.

However, it seems that the issue has been resolved with the help of another user who shared a link to a code implementation that resolved the problem.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your contribution to the LangChain project!