langchainjs icon indicating copy to clipboard operation
langchainjs copied to clipboard

LLMChain losing context

Open shoptacle opened this issue 2 years ago • 1 comments

The quickstart example for Model + Prompt = LLMChain, which should demonstrate message accumulation, isn't remembering (or at least not referencing) the previous input.

Not sure whether I'm doing something wrong or it's a regression. I've copied this excerpt from the doc example linked above:

  import { OpenAI } from 'langchain/llms/openai';
  import { PromptTemplate } from 'langchain/prompts';
  import { LLMChain } from 'langchain';

  const chat = new ChatOpenAI({ temperature: 0 });

  const translationPrompt = ChatPromptTemplate.fromPromptMessages([
    SystemMessagePromptTemplate.fromTemplate(
      'You are a helpful assistant that translates {input_language} to {output_language}.'
    ),
    HumanMessagePromptTemplate.fromTemplate('{text}'),
  ]);

  const chain = new LLMChain({
    prompt: translationPrompt,
    llm: chat,
  });

  const responseD = await chain.call({
    input_language: 'English',
    output_language: 'French',
    text: 'hi from London, how are you doing today',
  });
  console.log('responseD', JSON.stringify(responseD, null, 2));

  const responseE = await chain.call({
    input_language: 'English',
    output_language: 'French',
    text: 'Do you know where I am?',
  });
  console.log('responseE', JSON.stringify(responseE, null, 2));

EXPECTED OUTPUT (or similar reference to London): Yes, you mentioned that you are from London. However, as an AI language model, I don't have access to your current location unless you provide me with that information.

ACTUAL OUTPUT: I'm sorry, but as an AI language model, I don't have access to your location information. Is there anything else I can assist you with?

I've tried other references to information with similar results. Any ideas what I'm doing wrong here?

shoptacle avatar Apr 15 '23 00:04 shoptacle

+1 same

ucanlendit avatar Apr 23 '23 15:04 ucanlendit

Hi, @shoptacle! I'm here to help the LangChain team manage their backlog and I wanted to let you know that we are marking this issue as stale.

Based on my understanding, the issue you reported is about the LLMChain in the quickstart example not correctly referencing the previous input. It seems that the expected output should include a reference to London, but it is missing. Another user, @ucanlendit, has also experienced the same issue and added a +1 comment. However, there hasn't been any further activity or resolution reported.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project!

dosubot[bot] avatar Aug 19 '23 16:08 dosubot[bot]