[Model Request] please add "pkshatech/GLuCoSE-base-ja"

[xrd]

What happened?

I would like to use "pkshatech/GLuCoSE-base-ja"

What Python version are you on? e.g. python --version

Python3.10

Version

0.2.7 (Latest)

What os are you seeing the problem on?

Linux

Relevant stack traces and/or logs

No response

[quanduongduc]

You can manually add a custom model to the list of supported ONNX models in fastembed.text.onnx_embedding like this:

from fastembed.text.onnx_embedding import supported_onnx_models

supported_onnx_models.append({
    "model": "Alibaba-NLP/gte-large-en-v1.5",
    "dim": 1024,
    "description": "Custom Model",
    "size_in_GB": 1.7,
    "sources": {
        # from huggingface
        "hf": "Alibaba-NLP/gte-large-en-v1.5",
    },
    # This will make fastembed downloading this model file : https://huggingface.co/Alibaba-NLP/gte-large-en-v1.5/blob/main/onnx/model.onnx
    "model_file": "onnx/model.onnx",
})

# must be imported before the supported_onnx_models updated
from qdrant_client import QdrantClient, models

But it requires your model must be in ONNX format because fastembed uses the ONNX runtime. Also, make sure to include accurate values for dim, sources, and model_file.

I'm not sure about the accurate of embedding results but the model runs without any exceptions

[Anush008]

You can also PR the model to be added.

It could look like https://github.com/qdrant/fastembed/pull/270.

[joein]

@quanduongduc with this approach you would need to be careful with the postprocessing required by particular models, some models require pooling onnx output, some of them don't. You'd need to choose a proper implementation like onnx_embedding or pooled_embedding.

We also don't guarantee that these implementations won't be refactored without warnings, since we treat them as internal classes

No one restricts you from doing it though

[joein]

It is now possible to add models to fastembed in runtime via the official api)

However, it is still require the model to be in onnx and only supports a limited amount of postprocessing operations (mean pooling, cls embedding, no pooling, normalized output / not normalized)