Adding video to external library triggers full librabry retranscoding

[dronnikovigor]

The bug

Adding new video to external library triggers full librabry retranscoding. All videos were previously transcoded.

The OS that Immich Server is running on

OMV 7

Version of Immich Server

v1.115.0

Version of Immich Mobile App

v1.115.0

Platform with the issue

• [X] Server
• [ ] Web
• [ ] Mobile

Your docker-compose.yml content

version: '3.8'
name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - library:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - ${PHOTOS_PATH}:/mnt/media/library:ro
      - ${VIDEOS_PATH}:/mnt/media/videos:ro
    env_file:
      - stack.env
    devices:
      - /dev/dri:/dev/dri
    environment:
      - TZ=Canada/Eastern
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - stack.env
    restart: always

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
    volumes:
      - redis:/data

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_USER=${DB_USERNAME}
      - POSTGRES_DB=${DB_DATABASE_NAME}
      - POSTGRES_INITDB_ARGS=--data-checksums
    volumes:
      - postgres:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT SUM(checksum_failures) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

  backup:
    container_name: immich_db_dumper
    image: prodrigestivill/postgres-backup-local:14
    env_file:
      - stack.env
    environment:
      - POSTGRES_HOST=database
      - POSTGRES_CLUSTER=TRUE
      - POSTGRES_USER=${DB_USERNAME}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
      - POSTGRES_DB=${DB_DATABASE_NAME}
      - SCHEDULE=15 3 */3 * *
      - POSTGRES_EXTRA_OPTS=--clean --if-exists
      - BACKUP_DIR=/db_dumps
    volumes:
      - ${BACKUP_PATH}:/db_dumps
      - backup_data:/var/lib/postgresql/data
      - backups:/backups
    depends_on:
      - database


volumes:
  model-cache:
  redis:
  library:
  postgres:
  backup_data:
  backups:

Your .env content

-

Reproduction steps

1. Add video to external librabry
2. Check Jobs, Transcoding
3. All videos from librabry are in queue for transcoding.

Relevant log output

No response

Additional information

No response

[alextran1502]

@etnoy Hi Jon, have you seen this behavior?

[laurfb]

Somewhat. Any clip added (directly copied on an external folder or transferred by FTP) to the external library is transcoded several times. Initially, I thought that clips from the library were broken into chunks/pieces for transcoding, similar to a large file upload case. However, according to the log, it is simply transcoded, in full, several times. Even so, in the end, the clip preview is not available so we need to manually run the "Refresh encoded video" option to see a valid clip preview in browser. I have added the relevant sequence from the log file for some clip tests used ("test.mpg") copied in external library on Folder_1 folder. Please also notice, in the beginning of log, the error (already reported) - QueryFailedError: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"

immich v1.116.2

regards, Laur multiple encoding library 2.txt

[etnoy]

@etnoy Hi Jon, have you seen this behavior?

No, I'm not familiar with this. Let me look into it

[etnoy]

This should be fixed by #13251. Closing this for now, please reopen if it does not fix it.