haystack
haystack copied to clipboard
Add the memory feature to the library to enable chat + RAG
Add a memory abstraction to keep track of messages when running a chat pipeline that interacts with a retriever.
User story
Users should be able to create a pipeline that can answer user queries by using previous queries, responses (raw messages), as well as documents from a document store.
It needs to support the following conversation patterns:
App has access to previous content to understand context.
User: what is the tallest building in the US?
App: The North Tower.
User: How tall is it?
App: It is 417m long.
Outcome
Colab showcasing Chat + RAG.
### Tasks
- [ ] https://github.com/deepset-ai/haystack/issues/7868
- [ ] https://github.com/deepset-ai/haystack/issues/7917
- [ ] https://github.com/deepset-ai/haystack/issues/7918
- [ ] https://github.com/deepset-ai/haystack/issues/7919
- [ ] https://github.com/deepset-ai/haystack-cookbook/issues/97
There were some issues that were identified with the current proposed approach: https://github.com/deepset-ai/haystack-cookbook/pull/90#issuecomment-2161170220.
The two key issues today are:
- user messages are not tracked in memory
- the use of the
outputAdapter
component is bad for UX
We will gather requirements about how we want to be able to use the memory, and add them to this issue.
As a dev, I would like to:
- add chat messages to the memory from different parts of my graph (e.g. query input and from output of LLM) while keeping the additional number of components needed low
- have flexibility to store chat messages without the need for any conversion to documents. Ideally no conversion at all so that the memory can save chat messages directly.
@vblagoje Let's define what needs to be done in this issue together today. I have in mind breaking it up into four smaller issues and we can split the work.
- Add InMemoryChatMessageStore in haystack-experimental
- Add ChatMessageRetriever in haystack-experimental
- Add ChatMessageWriter in haystack-experimental
- Add user messages to memory in colab example notebook
Heyy, have a look at this issue #7830 where an abstraction for indexable objects is proposed. We could make ChatMessage
inherit from Indexable
and just by writing once the code for ObjectWriter
and for the different ObjectRetriver
components we can apply them to this use case.
Memory components were added to haystack-experimental and the cookbok repo contains a notebook showing Chat + RAG. Let's close this issue and focus instead on new tool abstractions as part of haystack-experimental. We're tracking the work on a notebook here https://github.com/deepset-ai/haystack/issues/8312