Enhanced Vector Store Capabilities with Full-Text/Hybrid Search and Reranking

[alessiobertazzo88]

trafficstars

• Updated the core vector store's classes to support vector search (enabled by default as today), full-text search, and reranking. Hybrid search is enabled when vector and full-text search are enabled at the same time.
• Added new default methods that throw UnsupportedOperationException to prevent the need for implementation across all vector stores unless specifically required.
• Implemented full-text search, hybrid search, and reranking functionalities specifically for the Azure Search AI vector store.

These enhancements improve the flexibility and functionality of the vector store classes, ensuring compatibility with various search and ranking operations while maintaining backward compatibility.

[tzolov]

Thank you @alessiobertazzo88 , This is very interesting.

May you which of the supported Vector Stores provide hybrid search as well?

[markpollack]

I was just about to merge this for M2 (even wrote docs) but then thought it might be better to push these variations into the search request and rename 'similaritySearch' to just 'search'. There are also other improvements to the VectorStore interface that should be done (for example options wrt to add), so I'll hold off merging for now.

[alessiobertazzo88]

@tzolov Yes, from a brief research I saw that other Vector Stores provide this feature

@markpollack ok I'm going to add some things. If you have some specific propose I'm glad to integrate it.

[johnsonr]

Agree re hybrid search. Reranking perhaps should be an advisor though.

[markpollack]

yes, reranking is a perfect use case for an advisor

[markpollack]

This will need to wait to post 1.0 GA.

[aadnegture]

when will this be prioritized?

[sobychacko]

@aadnegture We are actively working on a design for this, with the hope of incorporating it into the 1.1.0 version.