huggingface
Any plan to support qwen3 reranker model in TEI
Model description
HI, TEI. I am working with real-world customer i.e. ISV and startup to deploy RAG system. As qwen3. reranker is becoming popular, customers are asking if TEI can support this serving. Fundamentally this is a decoding model and need extra prompt to initialize the task request which is unlike old-school reranker model.
Open source status
- [x] The model implementation is available
- [x] The model weights are available
Provide useful links for the implementation
No response
Hey @tinafengfun, thanks for opening the issue! Indeed we do plan to support it and there are already a couple of PRs adding support for it (I still need to review those), but given that LLM-based reranking is rather new, that implies some core changes, so supporting those might not be straight forward (unless any of the aforementioned PRs do solve those things already). In any case, we'll work on it, and ping back in the relevant issues! 🤗
In the meantime, I'd suggest you to check Sentence Transformers to use Qwen3-Reranker models in the meantime if that helps, in any case, expect those to be available here soon!
just added the support in this PR. check it out please https://github.com/huggingface/text-embeddings-inference/pull/695
