djl icon indicating copy to clipboard operation
djl copied to clipboard

Add support for 'rotary' position embedding type for xml-roberta models

Open Aelentel opened this issue 1 year ago • 0 comments

Description

After converting the hugging face model "jinaai/jina-embeddings-v3" via djl-convert and patching the missing model type to "xml-roberta" it still barf on the unknown position embedding type.

  • Rust conversion didn't crash but didn't yield an usable model.
  • PyTorch crashed
  • OnnxRuntime crashed

version of djl-convert tested : 0.30.0 and 0.31.0

Will this change the current api? : absolutely no idea.

Who will benefit from this enhancement? any person using xml-roberta with "rotary" position embedding type

also this model is VERY efficient at embedding multilanguages and cross language semantic, i tested it on 20k docs on 13 different languages and the semantic matching was awesome, it'll be very good to support it.

References

  • works using the following code on python langchain.
from langchain_huggingface import HuggingFaceEmbeddings

model_name = "jinaai/jina-embeddings-v3"
model_kwargs = {"device": "cuda","trust_remote_code":True}

hf = HuggingFaceEmbeddings(model_name=model_name, model_kwargs=model_kwargs)

i'm pretty new on AI models so feel free to fix any of my mistakes :)

Aelentel avatar Nov 17 '24 20:11 Aelentel