infinity icon indicating copy to clipboard operation
infinity copied to clipboard

support Qwen3 embedding

Open bino282 opened this issue 6 months ago • 6 comments

bino282 avatar Jun 13 '25 01:06 bino282

Qwen3 embedding is ok. But not qwen3 reranker

BobLiu20 avatar Jun 19 '25 04:06 BobLiu20

Try an adapted version like this:

infinity_emb v2 --model-id tomaarsen/Qwen3-Reranker-0.6B-seq-cls

Note: You might be required to update your transformers version.

wirthual avatar Jun 21 '25 00:06 wirthual

With Transformers versions earlier than 4.51.0, you may encounter the following error: KeyError: 'qwen3'

Torala avatar Jun 22 '25 12:06 Torala

I run qwen3-reranker with the following commands:

# Thanks @Torala 
uv add "transformers>=4.51.0"

# https://github.com/huggingface/optimum/issues/2277
sed -i 's/raise RuntimeError/print/g' $(fd -HI __init__.py | grep 'bettertransformer/_')

# Thanks @wirthual
uv run infinity_emb v2 --model-id "tomaarsen/Qwen3-Reranker-0.6B-seq-cls"

fanyang89 avatar Jun 25 '25 10:06 fanyang89

+1 for reranker

dotmobo avatar Jun 28 '25 13:06 dotmobo