infinity
infinity copied to clipboard
support Qwen3 embedding
Qwen3 embedding is ok. But not qwen3 reranker
Try an adapted version like this:
infinity_emb v2 --model-id tomaarsen/Qwen3-Reranker-0.6B-seq-cls
Note: You might be required to update your transformers version.
With Transformers versions earlier than 4.51.0, you may encounter the following error: KeyError: 'qwen3'
I run qwen3-reranker with the following commands:
# Thanks @Torala
uv add "transformers>=4.51.0"
# https://github.com/huggingface/optimum/issues/2277
sed -i 's/raise RuntimeError/print/g' $(fd -HI __init__.py | grep 'bettertransformer/_')
# Thanks @wirthual
uv run infinity_emb v2 --model-id "tomaarsen/Qwen3-Reranker-0.6B-seq-cls"
+1 for reranker