haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Add the memory feature to the library to enable chat + RAG

Open mrm1001 opened this issue 9 months ago • 4 comments

Add a memory abstraction to keep track of messages when running a chat pipeline that interacts with a retriever.

User story

Users should be able to create a pipeline that can answer user queries by using previous queries, responses (raw messages), as well as documents from a document store.

It needs to support the following conversation patterns:

App has access to previous content to understand context.

User: what is the tallest building in the US?
App: The North Tower.
User: How tall is it?
App: It is 417m long.

Outcome

Colab showcasing Chat + RAG.

### Tasks
- [ ] https://github.com/deepset-ai/haystack/issues/7868
- [ ] https://github.com/deepset-ai/haystack/issues/7917
- [ ] https://github.com/deepset-ai/haystack/issues/7918
- [ ] https://github.com/deepset-ai/haystack/issues/7919
- [ ] https://github.com/deepset-ai/haystack-cookbook/issues/97

mrm1001 avatar May 02 '24 08:05 mrm1001

There were some issues that were identified with the current proposed approach: https://github.com/deepset-ai/haystack-cookbook/pull/90#issuecomment-2161170220.

The two key issues today are:

  • user messages are not tracked in memory
  • the use of the outputAdapter component is bad for UX

We will gather requirements about how we want to be able to use the memory, and add them to this issue.

mrm1001 avatar Jun 14 '24 12:06 mrm1001

As a dev, I would like to:

  • add chat messages to the memory from different parts of my graph (e.g. query input and from output of LLM) while keeping the additional number of components needed low
  • have flexibility to store chat messages without the need for any conversion to documents. Ideally no conversion at all so that the memory can save chat messages directly.

mrm1001 avatar Jun 21 '24 11:06 mrm1001

@vblagoje Let's define what needs to be done in this issue together today. I have in mind breaking it up into four smaller issues and we can split the work.

  1. Add InMemoryChatMessageStore in haystack-experimental
  2. Add ChatMessageRetriever in haystack-experimental
  3. Add ChatMessageWriter in haystack-experimental
  4. Add user messages to memory in colab example notebook

julian-risch avatar Jun 24 '24 07:06 julian-risch

Heyy, have a look at this issue #7830 where an abstraction for indexable objects is proposed. We could make ChatMessage inherit from Indexable and just by writing once the code for ObjectWriter and for the different ObjectRetriver components we can apply them to this use case.

CarlosFerLo avatar Jun 30 '24 09:06 CarlosFerLo

Memory components were added to haystack-experimental and the cookbok repo contains a notebook showing Chat + RAG. Let's close this issue and focus instead on new tool abstractions as part of haystack-experimental. We're tracking the work on a notebook here https://github.com/deepset-ai/haystack/issues/8312

julian-risch avatar Sep 03 '24 08:09 julian-risch