immich icon indicating copy to clipboard operation
immich copied to clipboard

exifTags.ModifyDate?.toDate is not a function

Open Hixxey opened this issue 1 year ago • 8 comments

The bug

I'm getting the below error when extracting metadata on new videos. I noticed that the thumbnails hadn't generated, so I checked the logs and found the below. I can run the thumbnail task manually but the metadata isn't extracting.

[Nest] 6 - 09/13/2024, 10:48:26 AM ERROR [Microservices:JobService] Unable to run job handler (metadataExtraction/metadata-extraction): TypeError: exifTags.ModifyDate?.toDate is not a function [Nest] 6 - 09/13/2024, 10:48:26 AM ERROR [Microservices:JobService] TypeError: exifTags.ModifyDate?.toDate is not a function at MetadataService.getDates (/usr/src/app/dist/services/metadata.service.js:508:46) at MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:194:80) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async /usr/src/app/dist/services/job.service.js:148:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 6 - 09/13/2024, 10:48:26 AM ERROR [Microservices:JobService] Object: { "id": "e8088678-a541-4abf-86ce-f077273bb3b0" }

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

v1.115.0

Version of Immich Mobile App

v1.115.0

Platform with the issue

  • [X] Server
  • [X] Web
  • [ ] Mobile

Your docker-compose.yml content

immich:
    container_name: immich
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    #command: [ "start.sh", "immich" ]
    volumes:
      - ./immich/library:/usr/src/app/upload/library
      - ./immich/cache/upload:/usr/src/app/upload/upload
      - ./immich/cache/thumbs:/usr/src/app/upload/thumbs
      - ./immich/cache/profile:/usr/src/app/upload/profile
      - ./immich/cache/encoded-video:/usr/src/app/upload/encoded-video
      - /etc/localtime:/etc/localtime:ro
      - ${HOMEDATA}/Pictures:/photos:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    environment:
      DB_USERNAME: ${ADMIN_USER}
      DB_PASSWORD: ${ADMIN_PASSWORD}
      DB_DATABASE_NAME: immich
      DB_HOSTNAME: immich_postgres
      REDIS_HOSTNAME: immich_redis
      TZ: ${TIMEZONE}
    depends_on:
      - immich_redis
      - immich_postgres
    restart: always
    labels:
      autoheal-app: true
  
  immich-machine-learning: #needs to be hyphenated otherwise the url doesn't work
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - ./immich/library:/usr/src/app/upload/library
      - ./immich/cache/upload:/usr/src/app/upload/upload
      - ./immich/cache/thumbs:/usr/src/app/upload/thumbs
      - ./immich/cache/profile:/usr/src/app/upload/profile
      - ./immich/cache/encoded-video:/usr/src/app/upload/encoded-video
      - ./immich-machine-learning/model-cache:/cache
    restart: always
    environment:
      TZ: ${TIMEZONE}
    labels:
      autoheal-app: true
  
  immich_redis:
    image: docker.io/redis:6.2-alpine@sha256:d6c2911ac51b289db208767581a5d154544f2b2fe4914ea5056443f62dc6e900
    container_name: immich_redis
    restart: always
    environment:
      TZ: ${TIMEZONE}
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - ./immich-redis:/data
    labels:
      autoheal-app: true
  
  immich_postgres:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${ADMIN_PASSWORD}
      POSTGRES_USER: ${ADMIN_USER}
      POSTGRES_DB: immich
      POSTGRES_INITDB_ARGS: '--data-checksums'
    ports:
      - "5432:5432"
    volumes:
      - ./immich-postgres:/var/lib/postgresql/data
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always
    labels:
      autoheal-app: true

Your .env content

IMMICH_VERSION=v1.115.0
ADMIN_USER=***
ADMIN_PASSWORD=*********
USER_ID=1000
GROUP_ID=1000
TIMEZONE="Europe/London"
HOMEDATA="/mnt/home"

Reproduction steps

  1. Upload video
  2. Refresh metadata
  3. View log

Relevant log output

[Nest] 6  - 09/13/2024, 10:48:26 AM   ERROR [Microservices:JobService] Unable to run job handler (metadataExtraction/metadata-extraction): TypeError: exifTags.ModifyDate?.toDate is not a function
[Nest] 6  - 09/13/2024, 10:48:26 AM   ERROR [Microservices:JobService] TypeError: exifTags.ModifyDate?.toDate is not a function
    at MetadataService.getDates (/usr/src/app/dist/services/metadata.service.js:508:46)
    at MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:194:80)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 6  - 09/13/2024, 10:48:26 AM   ERROR [Microservices:JobService] Object:
{
  "id": "e8088678-a541-4abf-86ce-f077273bb3b0"
}

Additional information

No response

Hixxey avatar Sep 13 '24 10:09 Hixxey

Some additional context, if it helps. The videos that are failing are GoPro videos that have been stitched together using Exiftool. I then edit them to make the DateTaken property the previous day. I've been doing this for months and Immich hasn't had any issues with the workflow until v1.115.0. All other videos that I tested appear to refresh metadata fine.

Hixxey avatar Sep 13 '24 11:09 Hixxey

Is the file something you can share to help with troubleshooting the issue?

alextran1502 avatar Sep 13 '24 15:09 alextran1502

https://github.com/user-attachments/assets/4f416927-ed9e-41fd-9a32-48ff160b79d4

Hopefully this uploads ok, please let me know if you have any issues. I've cut the file down to 3 seconds and tested. I get the same error.

Hixxey avatar Sep 13 '24 15:09 Hixxey

Can you zip the file and add it?

radh21301 avatar Sep 13 '24 16:09 radh21301

Can you zip the file and add it?

No problem, it should be attached. test.zip

Hixxey avatar Sep 13 '24 17:09 Hixxey

Hey. I was able to recreate the issue on my end. So, I ran this command: used the exiftool '-AllDates' command to set the date for the video you sent. I am not entire sure why what I did works, but it does. Immich logs look good and I can see the thumbnails. Maybe someone else can chime in.

exiftool "-AllDates=2022:06:09 18:22:4" -r -P -api:QuicktimeUTC -api largefilesupport=1 -overwrite_original ./test.mp4

Or even this for that matter -> which seems to be a better option according to https://exiftool.org/forum/index.php?topic=9267.0

exiftool "-time:all=2024:09:12 15:49:22" -wm w ./test.mp4

Here is the exiftool comparison - Left (before my edit. The exact file you sent). Right (after the exif edit)

image --- image

exiftool_after.txt exiftool_before.txt

radh21301 avatar Sep 13 '24 18:09 radh21301

Hey. I was able to recreate the issue on my end. So, I ran this command: used the exiftool '-AllDates' command to set the date for the video you sent. I am not entire sure why what I did works, but it does. Immich logs look good and I can see the thumbnails. Maybe someone else can chime in.


exiftool "-AllDates=2022:06:09 18:22:4" -r -P -api:QuicktimeUTC -api largefilesupport=1 -overwrite_original ./test.mp4



Here is the exiftool comparison - Left (before my edit. The exact file you sent). Right (after the exif edit)

image
image

exiftool_after.txt

exiftool_before.txt

Thanks for looking in to this. I could add that flag and I'm sure it'll resolve the issue, I'm just concerned that a bug has crept in because this exact file worked on v1.114.0.

Hixxey avatar Sep 13 '24 19:09 Hixxey

Thanks for investigating, i can confirm that i have same error since v1.115.0 on extract metadata with some media files

BDeus avatar Sep 13 '24 22:09 BDeus

Perfect, thanks for sorting 😊

Hixxey avatar Sep 14 '24 10:09 Hixxey

FWIW I am seeing the same errors

[Nest] 7  - 09/16/2024, 3:01:47 AM   ERROR [Microservices:JobService] Unable to run job handler (metadataExtraction/metadata-extraction): TypeError: exifTags.ModifyDate?.toDate is not a function
[Nest] 7  - 09/16/2024, 3:01:47 AM   ERROR [Microservices:JobService] TypeError: exifTags.ModifyDate?.toDate is not a function
    at MetadataService.getDates (/usr/src/app/dist/services/metadata.service.js:508:46)
    at MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:194:80)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7  - 09/16/2024, 3:01:47 AM   ERROR [Microservices:JobService] Object:
{
  "id": "4fe82285-1280-4842-b9d9-56676ec2801a",
  "source": "upload"
}

[Nest] 7  - 09/16/2024, 3:01:49 AM   ERROR [Microservices:JobService] Unable to run job handler (metadataExtraction/metadata-extraction): TypeError: exifTags.ModifyDate?.toDate is not a function
[Nest] 7  - 09/16/2024, 3:01:49 AM   ERROR [Microservices:JobService] TypeError: exifTags.ModifyDate?.toDate is not a function
    at MetadataService.getDates (/usr/src/app/dist/services/metadata.service.js:508:46)
    at MetadataService.handleMetadataExtraction (/usr/src/app/dist/services/metadata.service.js:194:80)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7  - 09/16/2024, 3:01:49 AM   ERROR [Microservices:JobService] Object:
{
  "id": "ee602546-2f94-4ebf-aeee-6dfed8954099",
  "source": "upload"
}

I don't plan to mess with my original files to rewrite their EXIF data. I would hope that if exif is not available, a timestamp from the file itself would be used?

I am also on v1.115.0. Looks like the fix was checked in / merged, so it would come in 1.116?

psla avatar Sep 16 '24 03:09 psla

@psla It has been fixed and will be in the next release. When the next release come out, you can just run metadata extraction and thumbnail generation for the problem asset

alextran1502 avatar Sep 16 '24 03:09 alextran1502

Awesome, thanks for the response! (it took me a moment to realize that the fix was already merged, I shouldn't have bothered you).

psla avatar Sep 16 '24 03:09 psla

@psla No problem! Cheers!

alextran1502 avatar Sep 16 '24 04:09 alextran1502

Hello. @alextran1502 bug is still here, 1.116 didn't fix it. After this crash, extract metadata job stops and not handle other files. image

AlexanderIlyanok avatar Sep 27 '24 13:09 AlexanderIlyanok

Hello, I wanted to add that in my case (I was expecting the new release because of this bug) it solved successfully the issue, so maybe this is other thing or a corner case? (thanks!, btw)

isaac-aa avatar Sep 27 '24 13:09 isaac-aa

@AlexanderIlyanok can you help confirm that the server is on 1.116 and if you can attach the problematic image, that would be helpful

alextran1502 avatar Sep 27 '24 13:09 alextran1502

@AlexanderIlyanok can you help confirm that the server is on 1.116 and if you can attach the problematic image, that would be helpful

Hello Alex. Sorry, it seems my fault - pulled new image but cheked, 1.115 started. By the way, after metadata extraction, not all Live iPhone photos merged (photot and mov). Is it known issue?

AlexanderIlyanok avatar Sep 27 '24 14:09 AlexanderIlyanok