feat: support bi-encoder models in TransformerSimilarityRanker

[Amnah199]

Is your feature request related to a problem? Please describe. The current implementation of TransformerSimilarityRanker only supports cross-encoder models, which limits the use of bi-encoder models like ColBERT v2.0.

Related discussion: Colbert as reranker

Describe the solution you'd like Update the TransformerSimilarityRanker or creating a new component to support bi-encoder models like ColBERT.

Describe alternatives you've considered Leave the current implementation as is.

Additional context Similar implementation: Llama ColbertRerank

[peteriz]

fastRAG is an extension of Haystack and has a Bi-encoder similarity ranker. You're invited to check it out here. And also ColBERT and PLAID support ..

@julian-risch We will be happy to contribute the bi-encoder ranker upstream to Haystack.

[sjrl]

Hey @peteriz if you would still be up for it that would be greatly appreciated!

[Ryzhtus]

@sjrl Hey, I will take it from here. Wanted to contribute this feature for a while, but didn't have time :)

[Ryzhtus]

@julian-risch @sjrl I’ve done some preliminary research on the original issue. It turns out that SentenceTransformersSimilarityRanker was introduced, and TransformersSimilarityRanker will no longer receive updates. So I have two ideas in mind:

1. Add general support for bi-encoder models in SentenceTransformersSimilarityRanker. As far as I understand the code, it currently supports only cross-encoder models, which is often sufficient in typical use cases (but maybe could be useful for some of the users).
2. Add support for ColBERT models (starting with reranking, and possibly retrieval later). ColBERT seems like a promising option in certain scenarios, and their repository has over 3.5k stars. However, it’s not integrated into sentence-transformers, so separate support for their API would need to be implemented — either via the colbert library itself or through fastembed (please see an example here).

Would love to hear you opinion! I would probably vote for the second option using fastembed

[sjrl]

Hey @Ryzhtus thanks for picking this up! And you are correct about the state of TransformersSimilarityRanker. I think both options sound great!

A few comments on both:

For Option 1 I think it's fine to fold bi-encoder support into the SentenceTransformersSimilarityRanker as long as it doesn't become too complicated to support both Cross Encoders and Bi-encoders in the same component and if we can avoid adding any new dependencies.

For Option 2 adding support for ColBERT would be awesome! For this one I think it would make sense to add this to our existing fastembed integration here https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/fastembed.

[RafaelJohn9]

Hey @Ryzhtus , you still working on this issue ?

happy to assist or collaborate 🤝

[KushagraaWadhwa]

Hi! I'd like to work on this issue. I'm interested in implementing Option 2 (adding support for ColBERT models using fastembed as suggested by @Ryzhtus and @sjrl). I'll start by looking into the fastembed integration and working on the ColBERT reranking functionality. Happy to collaborate with @RafaelJohn9 or anyone else interested!

[RafaelJohn9]

Hi! I'd like to work on this issue. I'm interested in implementing Option 2 (adding support for ColBERT models using fastembed as suggested by @Ryzhtus and @sjrl). I'll start by looking into the fastembed integration and working on the ColBERT reranking functionality. Happy to collaborate with @RafaelJohn9 or anyone else interested!

Hey @akkefa ,

I haven't started working on the issue. You can have it🤝👍.