Unable to deploy Embedding model "hkunlp/instructor-xl"

[ranjanshivaji]

Trying to deploy the Embedding model "hkunlp/instructor-xl" Below is the Deployment file used with the model-id as args.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: instructor-xl-tei
  namespace: openllm2
  labels:
    app: instructor-xl-tei
spec:
  replicas: 1
  selector:
    matchLabels:
      app: instructor-xl-tei
  template:
    metadata:
      labels:
        app: instructor-xl-tei
    spec:
      containers:
      - name: instructor-xl-tei-container
        image: ghcr.io/huggingface/text-embeddings-inference:0.6
        args:
        - "--model-id"
        - "hkunlp/instructor-xl"
        - "--revision"
        - "refs/pr/6"
        ports:
        - containerPort: 8080
        resources:
          limits:
            memory: "28Gi"
            cpu: "6"
            nvidia.com/gpu: 1 

But the Text Embedding Image spitting out the below error:

2023-12-20T08:47:19.785313Z  INFO text_embeddings_router: router/src/main.rs:112: Args { model_id: "hku***/*********r-xl", revision: Some("refs/pr/6"), tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 512, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, hf_api_token: None, hostname: "instructor-xl-tei-7799f985f6-vjmt6", port: 80, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: Some("/data"), json_output: false, otlp_endpoint: None }
2023-12-20T08:47:19.785391Z  INFO hf_hub: /root/.cargo/git/checkouts/hf-hub-1aadb4c6e2cbe1ba/b167f69/src/lib.rs:55: Token file not found "/root/.cache/huggingface/token"    
2023-12-20T08:47:20.163267Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:9: Starting download
2023-12-20T08:47:32.234688Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:26: Model artifacts downloaded in 12.07142108s
2023-12-20T08:47:32.296733Z  INFO text_embeddings_core::tokenization: core/src/tokenization.rs:22: Starting 24 tokenization workers
2023-12-20T08:47:32.712214Z  INFO text_embeddings_router: router/src/lib.rs:239: Starting model backend
Error: Could not create backend

Caused by:
    Could not start backend: missing field `hidden_size` at line 60 column 1

Is the model Not supported or if there is something wrong in the deployment steps or arguments.

[OlivierDehaene]

T5 models like instructor are not yet supported.