Support for v2 version of Mixedbread Reranker Base

[axeloh]

Model description

Hi there :)

I have been using the mixedbread-ai/mxbai-rerank-base-v1 served with Infinity via Runpod for some time now. However, mixedbread has released a v2 version: https://huggingface.co/mixedbread-ai/mxbai-rerank-base-v2,

for which I get the following error when trying to use in the same way I am using v1:

{ "error": { "message": "ModelNotDeployedError: model=mixedbread-ai/mxbai-rerank-base-v2does not supportrerank. Reason: the loaded moded cannot fullyfill rerank. Options are {'embed'}.", "type": null, "param": null, "code": 400 } }

Open source status & huggingface transformers.

• [ ] The model implementation is available on transformers
• [ ] The model weights are available on huggingface-hub
• [ ] I verified that the model is currently not running in the latest version pip install infinity_emb[all] --upgrade
• [ ] I made the authors of the model aware that I want to use it with infinity_emb & check if they are aware of the issue.

[michaelfeil]

Hey there!

I am working on getting mixedbread up as a classification model. https://huggingface.co/michaelfeil/mxbai-rerank-large-v2-seq

[axeloh]

Thanks for the quick reply @michaelfeil!

Classification as in a "reranker classification" labeling as 0 (not relevant) or 1 (relevant)? What about the option to use it as a regular reranker?

Thanks :)

[axeloh]

Hello again @michaelfeil! :) Got it working as a reranker in the end using your adaption of the model as a classification model. However, any plans on adding support for it as a "native" reranker via infinity (as the v1 version)?

[michaelfeil]

Not for now, the reason is: Its a classification model, not reranker