serving bge-reranker-v2-m3

[gree2]

System Info

ghcr.io/huggingface/text-embeddings-inference:cpu-1.5

Information

• [X] Docker
• [ ] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

docker run command

docker run --name bge_rrk_6201 -d -p 6201:80 -v /data/sdd/models:/data ghcr.io/huggingface/text-embeddings-inference:cpu-1.5 --model-id /data/bge-reranker-v2-m3

docker logs command

2024-09-14T10:45:19.877443Z  INFO text_embeddings_router: router/src/main.rs:175: Args { model_id: "/dat*/***-********-*2-m3", revision: None, tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 512, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, auto_truncate: false, default_prompt_name: None, default_prompt: None, hf_api_token: None, hostname: "2774d18b0909", port: 80, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: Some("/data"), payload_limit: 2000000, api_key: Some("sk-aaabbbcccdddeeefffggghhhiiijjjkkk"), json_output: false, otlp_endpoint: None, otlp_service_name: "text-embeddings-inference.server", cors_allow_origin: None }
2024-09-14T10:45:20.605567Z  WARN text_embeddings_router: router/src/lib.rs:195: Could not find a Sentence Transformers config
2024-09-14T10:45:20.605594Z  INFO text_embeddings_router: router/src/lib.rs:199: Maximum number of tokens per request: 8192
2024-09-14T10:45:20.606233Z  INFO text_embeddings_core::tokenization: core/src/tokenization.rs:28: Starting 32 tokenization workers
2024-09-14T10:45:33.019924Z  INFO text_embeddings_router: router/src/lib.rs:241: Starting model backend
thread '<unnamed>' panicked at backends/ort/src/lib.rs:363:30:
no entry found for key
stack backtrace:
   0:     0x557f0f47be4c - <unknown>
   1:     0x557f0f147080 - <unknown>
   2:     0x557f0f4492a2 - <unknown>
   3:     0x557f0f47d9fe - <unknown>
   4:     0x557f0f47d170 - <unknown>
   5:     0x557f0f47e332 - <unknown>
   6:     0x557f0f47dd5c - <unknown>
   7:     0x557f0f47dcb6 - <unknown>
   8:     0x557f0f47dca1 - <unknown>
   9:     0x557f0ed04534 - <unknown>
  10:     0x557f0ed04b12 - <unknown>
  11:     0x557f0f2a4d6f - <unknown>
  12:     0x557f0f4bc820 - <unknown>
  13:     0x557f0f482ba9 - <unknown>
  14:     0x557f0f481a4d - <unknown>
  15:     0x557f0f47efe5 - <unknown>
  16:     0x7f6773a5c134 - <unknown>
  17:     0x7f6773adba40 - clone
  18:                0x0 - <unknown>

Expected behavior

Does tei support bge-reranker-v2-m3 or not? I can use tei to serving bge-m3.