immich
immich copied to clipboard
Published
20 hours ago •
immich-app
immich-app
OpenVino: facial detection is broken with 1.99.0
The bug
When I try launching the face detection, whatever model I use, I get the following error:
immich_machine_learning | [03/23/24 22:46:05] INFO Setting 'buffalo_l' execution providers to
immich_machine_learning | ['OpenVINOExecutionProvider',
immich_machine_learning | 'CPUExecutionProvider'], in descending order of
immich_machine_learning | preference
immich_machine_learning | [03/23/24 22:46:05] INFO Loading facial recognition model 'buffalo_l' to
immich_machine_learning | memory
immich_machine_learning | [03/23/24 22:46:08] ERROR Exception in ASGI application
immich_machine_learning |
immich_machine_learning | ╭─────── Traceback (most recent call last) ───────╮
immich_machine_learning | │ /usr/src/app/main.py:118 in predict │
immich_machine_learning | │ │
immich_machine_learning | │ 115 │ │
immich_machine_learning | │ 116 │ model = await load(await model_cache. │
immich_machine_learning | │ ttl=settings.model_ttl, **kwargs)) │
immich_machine_learning | │ 117 │ model.configure(**kwargs) │
immich_machine_learning | │ ❱ 118 │ outputs = await run(model.predict, in │
immich_machine_learning | │ 119 │ return ORJSONResponse(outputs) │
immich_machine_learning | │ 120 │
immich_machine_learning | │ 121 │
immich_machine_learning | │ │
immich_machine_learning | │ /usr/src/app/main.py:125 in run │
immich_machine_learning | │ │
immich_machine_learning | │ 122 async def run(func: Callable[..., Any], i │
immich_machine_learning | │ 123 │ if thread_pool is None: │
immich_machine_learning | │ 124 │ │ return func(inputs) │
immich_machine_learning | │ ❱ 125 │ return await asyncio.get_running_loop │
immich_machine_learning | │ 126 │
immich_machine_learning | │ 127 │
immich_machine_learning | │ 128 async def load(model: InferenceModel) -> │
immich_machine_learning | │ │
immich_machine_learning | │ /usr/lib/python3.10/concurrent/futures/thread.p │
immich_machine_learning | │ y:58 in run │
immich_machine_learning | │ │
immich_machine_learning | │ /usr/src/app/models/base.py:59 in predict │
immich_machine_learning | │ │
immich_machine_learning | │ 56 │ │ self.load() │
immich_machine_learning | │ 57 │ │ if model_kwargs: │
immich_machine_learning | │ 58 │ │ │ self.configure(**model_kwargs │
immich_machine_learning | │ ❱ 59 │ │ return self._predict(inputs) │
immich_machine_learning | │ 60 │ │
immich_machine_learning | │ 61 │ @abstractmethod │
immich_machine_learning | │ 62 │ def _predict(self, inputs: Any) -> An │
immich_machine_learning | │ │
immich_machine_learning | │ /usr/src/app/models/facial_recognition.py:49 in │
immich_machine_learning | │ _predict │
immich_machine_learning | │ │
immich_machine_learning | │ 46 │ │ else: │
immich_machine_learning | │ 47 │ │ │ decoded_image = image │
immich_machine_learning | │ 48 │ │ assert is_ndarray(decoded_image, n │
immich_machine_learning | │ ❱ 49 │ │ bboxes, kpss = self.det_model.dete │
immich_machine_learning | │ 50 │ │ if bboxes.size == 0: │
immich_machine_learning | │ 51 │ │ │ return [] │
immich_machine_learning | │ 52 │ │ assert is_ndarray(kpss, np.float32 │
immich_machine_learning | │ │
immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning | │ ace/model_zoo/retinaface.py:224 in detect │
immich_machine_learning | │ │
immich_machine_learning | │ 221 │ │ det_img = np.zeros( (input_size[1 │
immich_machine_learning | │ 222 │ │ det_img[:new_height, :new_width, │
immich_machine_learning | │ 223 │ │ │
immich_machine_learning | │ ❱ 224 │ │ scores_list, bboxes_list, kpss_li │
immich_machine_learning | │ 225 │ │ │
immich_machine_learning | │ 226 │ │ scores = np.vstack(scores_list) │
immich_machine_learning | │ 227 │ │ scores_ravel = scores.ravel() │
immich_machine_learning | │ │
immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/insightf │
immich_machine_learning | │ ace/model_zoo/retinaface.py:152 in forward │
immich_machine_learning | │ │
immich_machine_learning | │ 149 │ │ kpss_list = [] │
immich_machine_learning | │ 150 │ │ input_size = tuple(img.shape[0:2] │
immich_machine_learning | │ 151 │ │ blob = cv2.dnn.blobFromImage(img, │
immich_machine_learning | │ (self.input_mean, self.input_mean, self.i │
immich_machine_learning | │ ❱ 152 │ │ net_outs = self.session.run(self. │
immich_machine_learning | │ 153 │ │ │
immich_machine_learning | │ 154 │ │ input_height = blob.shape[2] │
immich_machine_learning | │ 155 │ │ input_width = blob.shape[3] │
immich_machine_learning | │ │
immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:22 │
immich_machine_learning | │ 0 in run │
immich_machine_learning | │ │
immich_machine_learning | │ 217 │ │ if not output_names: │
immich_machine_learning | │ 218 │ │ │ output_names = [output.name │
immich_machine_learning | │ 219 │ │ try: │
immich_machine_learning | │ ❱ 220 │ │ │ return self._sess.run(output │
immich_machine_learning | │ 221 │ │ except C.EPFail as err: │
immich_machine_learning | │ 222 │ │ │ if self._enable_fallback: │
immich_machine_learning | │ 223 │ │ │ │ print(f"EP Error: {err!s │
immich_machine_learning | ╰─────────────────────────────────────────────────╯
immich_machine_learning | RuntimeException: [ONNXRuntimeError] : 6 :
immich_machine_learning | RUNTIME_EXCEPTION : Encountered unknown exception
immich_machine_learning | in Run()
immich_microservices | [Nest] 7 - 03/23/2024, 10:46:08 PM ERROR [JobService] Unable to run job handler (faceDetection/face-detection): Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices | [Nest] 7 - 03/23/2024, 10:46:08 PM ERROR [JobService] Error: Machine learning request for facial recognition failed with status 500: Internal Server Error
immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/infra/repositories/machine-learning.repository.js:23:19)
immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices | at async PersonService.handleDetectFaces (/usr/src/app/dist/domain/person/person.service.js:248:23)
immich_microservices | at async /usr/src/app/dist/domain/job/job.service.js:137:36
immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_microservices | [Nest] 7 - 03/23/2024, 10:46:08 PM ERROR [JobService] Object:
immich_microservices | {
immich_microservices | "id": "9e1d4bbf-84c2-40dd-9aec-c913e5a1a662"
immich_microservices | }
immich_microservices |
Regular Smart Search proceeds without issue.
The OS that Immich Server is running on
Proxmox 8.1 (6.5 Linux Kernel)
Version of Immich Server
1.99.0
Version of Immich Mobile App
1.99.0
Platform with the issue
- [X] Server
- [ ] Web
- [ ] Mobile
Your docker-compose.yml content
#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#
name: immich
services:
immich-server:
container_name: immich_server
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
command: [ "start.sh", "immich" ]
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
- /mount/op11:/usr/src/app/external
- /etc/localtime:/etc/localtime:ro
env_file:
- .env
ports:
- 2283:3001
depends_on:
- redis
- database
restart: always
immich-microservices:
container_name: immich_microservices
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
extends:
file: hwaccel.transcoding.yml
service: quicksync # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
command: [ "start.sh", "microservices" ]
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
- /mount/op11:/usr/src/app/external
- /etc/localtime:/etc/localtime:ro
env_file:
- .env
depends_on:
- redis
- database
restart: always
immich-machine-learning:
container_name: immich_machine_learning
# For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
# Example tag: ${IMMICH_VERSION:-release}-cuda
image: ghcr.io/immich-app/immich-machine-learning:main-openvino
extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
file: hwaccel.ml.yml
service: openvino # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
volumes:
- model-cache:/cache
env_file:
- .env
restart: always
redis:
container_name: immich_redis
image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
restart: always
database:
container_name: immich_postgres
image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_DB: ${DB_DATABASE_NAME}
volumes:
- pgdata:/var/lib/postgresql/data
restart: always
volumes:
pgdata:
model-cache:
Your .env content
# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables
# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release
# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=(redacted)
# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
REDIS_HOSTNAME=immich_redis
Reproduction steps
1. docker compose up
2. --> Jobs
3. --> Click Face Detection: All
...
Additional information
My processor is an Intel N100. Previously to 1.99.0, the face detection was working, but I had issues with the smart search, so I guess it's hard to get all of it with OpenVino haha.
Thank you for the gigantic work up to now! Leo
