ConversationalRetrievalChain unable to use ConversationBufferMemory when return_source_documents=True

[rob-rowe]

System Info

LangChain 0.0.188 Python 3.8

Who can help?

@hwchase17 @agola11

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [ ] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [ ] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [X] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True) qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature =0), vectorstore.as_retriever(), memory=memory, return_source_documents=True) qa({"question": query}) #ValueError" One output key expected, got dict_keys(['answer', 'source_documents'])

qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature =0), vectorstore.as_retriever(), return_source_documents=True) qa({"question": query, "chat_history": []}) #...(runs as expected)

Expected behavior

ConversationalRetrievalChain does not work with ConversationBufferMemory and return_source_documents=True.

ValueError: One output key expected, got dict_keys(['answer', 'source_documents'])

[communiteq]

As a workaround, you can define this class and use it instead.

class AnswerConversationBufferMemory(ConversationBufferMemory):
    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
        return super(AnswerConversationBufferMemory, self).save_context(inputs,{'response': outputs['answer']})

[rob-rowe]

@communiteq Thanks. That worked

[Vaishnavi21721]

I am doing the same but still the same issue. Can you please help. This is my code from typing import Dict, Any class AnswerConversationBufferMemory(ConversationBufferMemory): def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None: return super(AnswerConversationBufferMemory, self).save_context(inputs,{'response': outputs['answer']}) history = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

chain = ConversationalRetrievalChain.from_llm( OpenAI(temperature=0.8), store.as_retriever(search_kwargs={"k": 3}), memory=history,get_chat_history=get_chat_history, return_source_documents=True ) q_1 = "What is a lead?" result = chain({"question": q_1})

result

[felipeff]

@Vaishnavi21721 you're defining the class but not using it. Change: history = ConversationBufferMemory(memory_key="chat_history", return_messages=True)

To: history = AnswerConversationBufferMemory(memory_key="chat_history", return_messages=True)

[Vaishnavi21721]

@felipeff yes, I already did that but not working.

[dolphinsting]

That worked for me. Thanks. For other people who come across this, it didn't work when I used chain.run(query) but did work for chain(query).

[sergeiAI]

As a workaround, you can define this class and use it instead.

class AnswerConversationBufferMemory(ConversationBufferMemory):
    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
        return super(AnswerConversationBufferMemory, self).save_context(inputs,{'response': outputs['answer']})

Thank you! This helped me as well. Why don't any of the renowned chatbots, such as ChatGPT or Bard, help with these kinds of questions? I've noticed it's a total waste of time, as of now. I still get my money's worth out of Stack Overflow and GitHub.

[bllchmbrs]

Still a bug in

import langchain
langchain.__version__
'0.0.251'

Another workaround is...

def get_qa_with_sources_chain():
    llm = ChatOpenAI(model_name="gpt-4", temperature=0)
    retriever = load_retriever()
    history = []
    model = ConversationalRetrievalChain.from_llm(
        llm=llm,
        retriever=retriever,
        return_source_documents=True)
    # bug: this doesn't work with the built-in memory
    # hacking around it for the tutorial

    def model_func(question):
        new_input = {"question": question['question'], "chat_history": history}
        result = model(new_input)
        history.append((question['question'], result['answer']))
        return result

    return model_func
    ```

[dosubot[bot]]

Hi, @rob-rowe. I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you reported is related to the ConversationalRetrievalChain being unable to use ConversationBufferMemory when return_source_documents=True, resulting in a ValueError. User "communiteq" suggested a workaround by defining a new class called AnswerConversationBufferMemory and using it instead. This workaround was confirmed to work by you and other users. User "felipeff" also provided additional guidance to another user facing the same issue. User "bllchmbrs" reported a bug and provided another workaround.

Before we close this issue, we wanted to check if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project. Let us know if you have any further questions or concerns.