Flowise icon indicating copy to clipboard operation
Flowise copied to clipboard

[FEATURE] Pinecone: Retrieve X Chunks Before & Y Chunks after the Matching Chunk

Open mikehudson2 opened this issue 1 year ago • 2 comments

In the Flowise Vectara node, in response to a query, Vectara similarity matches the query to a single sentence chunk in the VDB, and retrieves that sentence. It can also optionally retrieve x sentences before and y sentences after the matching sentence. This is really useful for bringing in context wider than the matching chunk.

Would it be possible to do something similar in Pinecone or Weaviate so that the matching chunk is retrieved together with a specified number of contiguous chunks before and after it?

mikehudson2 avatar Feb 20 '24 23:02 mikehudson2

this is a functionality offered by Vectara, I'm not sure if Pinecone or Weaviate offer similar functionality

HenryHengZJ avatar Feb 22 '24 15:02 HenryHengZJ

I don't think P or W do this out-of-the-box. But something similar would be achievable using Langchain's Parent Document Retriever: https://python.langchain.com/docs/modules/data_connection/retrievers/parent_document_retriever. Any chance of that being added?

mikehudson2 avatar Feb 22 '24 16:02 mikehudson2