ml-commons
ml-commons copied to clipboard
opensearch-project
[BUG] Failed to deploy local ONNX model "gte-large-en-v1.5" with error "input mismatch"
Describe the bug
With Opensearch v2.15.0, When deploying local ONNX model "gte-large-en-v1.5", I got an error of:
input mismatch, looking for: [input_ids, attention_mask, token_type_ids]
The model zip file can be downloaded from: https://github.com/magda-io/embedding_models/releases/tag/gte-large-en-v1.5
The model can be registered & downloaded properly. But when deploy, the error above will be produced.
However, a smaller version of similar model "gte-base-en-v1.5" can be deployed (and create emebedings) properly with no issue.
The working model can be found here: https://github.com/magda-io/embedding_models/releases/tag/gte-base-en-v1.5
Related component
Plugins
To Reproduce
Following the guide here: https://opensearch.org/docs/latest/ml-commons-plugin/custom-local-models/
- Register a model group
- Register a local model
POST /_plugins/_ml/models/_register
{
"name": "gte-large-en-v1.5",
"version": "1.5.0",
"model_group_id": "wlcnb4kBJ1eYAeTMHlV6",
"description": "gte-large-en-v1.5",
"function_name": "TEXT_EMBEDDING",
"model_format": "ONNX",
"model_content_size_in_bytes": 1271086226,
"model_content_hash_value": "72e4037cd3e2a2499a3c92759159982982871a40715ca7c38f583934aa17306b",
"model_config": {
"model_type": "BertModel",
"embedding_dimension": 768,
"framework_type": "sentence_transformers"
},
"url": "https://github.com/magda-io/embedding_models/releases/download/gte-large-en-v1.5/model.zip"
}
- check the status of the register operation
- Deploy the model via endpoint POST /_plugins/_ml/models/[mode id]/_deploy
- check the status of the deploy operation and you will see the fail status & error
Expected behavior
Be able to successfully deploy the ONNX format model without error.
Additional Details
Plugins Running the official Docker image with no additional plugins installed.
Screenshots If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
- OS: Running the official Docker image
- Version v2.15.0
Additional context Add any other context about the problem here.
Hello, Same error with "Snowflake/snowflake-arctic-embed-m" https://huggingface.co/Snowflake/snowflake-arctic-embed-m
