ML: Openvino Error: Machine learning request for clip failed with status 500: Internal Server Error

[Rhevin]

The bug

I am using a mini pc with an Intel N100 CPU to run immich with machine learning using openvino setup. I have installed openvino on my OS, but I experienced this error when I uploaded an image

immich_machine_learning  | [02/16/24 02:57:10] ERROR    Exception in ASGI application                      
immich_machine_learning  |                                                                                 
immich_machine_learning  |                              ╭─────── Traceback (most recent call last) ───────╮
immich_machine_learning  |                              │ /usr/src/app/main.py:108 in predict             │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   105 │                                         │
immich_machine_learning  |                              │   106 │   model = await load(await model_cache. │
immich_machine_learning  |                              │   107 │   model.configure(**kwargs)             │
immich_machine_learning  |                              │ ❱ 108 │   outputs = await run(model.predict, in │
immich_machine_learning  |                              │   109 │   return ORJSONResponse(outputs)        │
immich_machine_learning  |                              │   110                                           │
immich_machine_learning  |                              │   111                                           │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:115 in run                 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   112 async def run(func: Callable[..., Any], i │
immich_machine_learning  |                              │   113 │   if thread_pool is None:               │
immich_machine_learning  |                              │   114 │   │   return func(inputs)               │
immich_machine_learning  |                              │ ❱ 115 │   return await asyncio.get_running_loop │
immich_machine_learning  |                              │   116                                           │
immich_machine_learning  |                              │   117                                           │
immich_machine_learning  |                              │   118 async def load(model: InferenceModel) ->  │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/lib/python3.10/concurrent/futures/thread.p │
immich_machine_learning  |                              │ y:58 in run                                     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/base.py:61 in predict       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    58 │   │   self.load()                       │
immich_machine_learning  |                              │    59 │   │   if model_kwargs:                  │
immich_machine_learning  |                              │    60 │   │   │   self.configure(**model_kwargs │
immich_machine_learning  |                              │ ❱  61 │   │   return self._predict(inputs)      │
immich_machine_learning  |                              │    62 │                                         │
immich_machine_learning  |                              │    63 │   @abstractmethod                       │
immich_machine_learning  |                              │    64 │   def _predict(self, inputs: Any) -> An │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/clip.py:52 in _predict      │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    49 │   │   │   case Image.Image():           │
immich_machine_learning  |                              │    50 │   │   │   │   if self.mode == "text":   │
immich_machine_learning  |                              │    51 │   │   │   │   │   raise TypeError("Cann │
immich_machine_learning  |                              │ ❱  52 │   │   │   │   outputs: NDArray[np.float │
immich_machine_learning  |                              │       self.transform(image_or_text))[0][0]      │
immich_machine_learning  |                              │    53 │   │   │   case str():                   │
immich_machine_learning  |                              │    54 │   │   │   │   if self.mode == "vision": │
immich_machine_learning  |                              │    55 │   │   │   │   │   raise TypeError("Cann │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
immich_machine_learning  |                              │ ime/capi/onnxruntime_inference_collection.py:21 │
immich_machine_learning  |                              │ 9 in run                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   216 │   │   if not output_names:              │
immich_machine_learning  |                              │   217 │   │   │   output_names = [output.name f │
immich_machine_learning  |                              │   218 │   │   try:                              │
immich_machine_learning  |                              │ ❱ 219 │   │   │   return self._sess.run(output_ │
immich_machine_learning  |                              │   220 │   │   except C.EPFail as err:           │
immich_machine_learning  |                              │   221 │   │   │   if self._enable_fallback:     │
immich_machine_learning  |                              │   222 │   │   │   │   print(f"EP Error: {str(er │
immich_machine_learning  |                              ╰─────────────────────────────────────────────────╯
immich_machine_learning  |                              Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero     
immich_machine_learning  |                              status code returned while running                 
immich_machine_learning  |                              OpenVINO-EP-subgraph_1 node.                       
immich_machine_learning  |                              Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgrap
immich_machine_learning  |                              h_1_0' Status Message:                             
immich_machine_learning  |                              /home/onnxruntimedev/onnxruntime/onnxruntime/core/p
immich_machine_learning  |                              roviders/openvino/ov_interface.cc:53               
immich_machine_learning  |                              onnxruntime::openvino_ep::OVExeNetwork             
immich_machine_learning  |                              onnxruntime::openvino_ep::OVCore::LoadNetwork(const
immich_machine_learning  |                              string&, std::string&, ov::AnyMap&, std::string)   
immich_machine_learning  |                              [OpenVINO-EP]  Exception while Loading Network for 
immich_machine_learning  |                              graph:                                             
immich_machine_learning  |                              OpenVINOExecutionProvider_OpenVINO-EP-subgraph_1_0C
immich_machine_learning  |                              heck 'false' failed at                             
immich_machine_learning  |                              src/inference/src/core.cpp:149:                    
immich_machine_learning  |                              Check 'false' failed at                            
immich_machine_learning  |                              src/frontends/common/src/frontend.cpp:53:          
immich_machine_learning  |                              Converting input model                             
immich_machine_learning  |                              Input for function node LayerNormalization_841 is  
immich_machine_learning  |                              out of bounds                                      
immich_machine_learning  |                                                                                 
immich_machine_learning  |                                                                                 
immich_machine_learning  |                                                                                 
immich_microservices     | [Nest] 7  - 02/16/2024, 2:57:10 AM   ERROR [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 500: Internal Server Error
immich_microservices     | [Nest] 7  - 02/16/2024, 2:57:10 AM   ERROR [JobService] Error: Machine learning request for clip failed with status 500: Internal Server Error

Is this related to openvino bug?

The OS that Immich Server is running on

Docker, CoreOS

Version of Immich Server

v1.94.1

Version of Immich Mobile App

N/A

Platform with the issue

• [X] Server
• [ ] Web
• [ ] Mobile

Your docker-compose.yml content

using default latest docker-compose except this part

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino
    extends:
       file: hwaccel.ml.yml
       service: openvino
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

Your .env content

N/A

Reproduction steps

1. use immich v1.94.1 docker-compose with openvino config including hwaccel.ml.yaml
2. upload image
3. check docker log

Additional information

No response

[mertalev]

This is being tracked in discussion #6869, but I'll leave this open since it appears to be a general issue with OpenVINO and not something model-specific.

[AudriusTGo]

have same problem

[snasui23]

I have the same issue with the same Intel N100 CPU, running docker in: Distributor ID: Debian Description: Debian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye

I have tried to run with and without -openvino tag without success.

In the end I disabled the Machine Learning option from settings and everything else runs as expected :).

[smtdev]

Same over here using v1.95.1. OpenVINO too.