immich [BUG] H265 videos not properly converted to H264

The bug

If I record a video with the default camera app on my Sony Xperia 5 II and then sync it to Immich, I cannot play the video in Firefox. I can play it fine in Edge/Chrome, but I cannot play it in Firefox/Waterfox Desktop or Mobile. If I open the file directly with Firefox, I only get audio. If I open the immich encoded video directly with Firefox, it doesn't play it at all (says file is corrupted).

The ffmpeg settings in immich are default: -crf 23 -preset ultrafast audio coded aac video codec h264 target resolution 720p max bitrate 0 threads 0 transcode only videos not in the desired format

Here are the original (~5.8 MB) file and the one that immich transcoded (~380 KB)

https://github.com/immich-app/immich/assets/26489428/3068b7ca-4ced-43f7-aa22-2594c4dbbc3a

https://github.com/immich-app/immich/assets/26489428/975bf794-7bc9-4271-a207-1b91b37cb4a5

The OS that Immich Server is running on

Linux Mint 21.1

Version of Immich Server

v1.65.0

Version of Immich Mobile App

1.65.0 build.88

Platform with the issue

[X] Server
[X] Web
[ ] Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    entrypoint: ["/bin/sh", "./start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:release
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:release
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:release
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

###################################################################################
# Database
###################################################################################

# NOTE: The following four database variables support Docker secrets by adding a *_FILE suffix to the variable name
# See the docker-compose documentation on secrets for additional details: https://docs.docker.com/compose/compose-file/compose-file-v3/#secrets
DB_HOSTNAME=immich_postgres
DB_USERNAME=<redacted>
DB_PASSWORD=<redacted>
DB_DATABASE_NAME=<redacted>

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# REDIS_URL will be used to pass custom options to ioredis.
# Example for Sentinel
# {"sentinels":[{"host":"redis-sentinel-node-0","port":26379},{"host":"redis-sentinel-node-1","port":26379},{"host":"redis-sentinel-node-2","port":26379}],"name":"redis-sentinel"}
# REDIS_URL=ioredis://<redacted>

# Optional Redis settings:

# Note: these parameters are not automatically passed to the Redis Container
# to do so, please edit the docker-compose.yml file as well. Redis is not configured
# via environment variables, only redis.conf or the command line

# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_USERNAME=
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=<redacted>


###################################################################################
# Typesense
###################################################################################
TYPESENSE_API_KEY=<redacted>
# TYPESENSE_ENABLED=false
# TYPESENSE_URL uses base64 encoding for the nodes json.
# Example JSON that was used:
# [
#      { 'host': 'typesense-1.example.net', 'port': '443', 'protocol': 'https' },
#      { 'host': 'typesense-2.example.net', 'port': '443', 'protocol': 'https' },
#      { 'host': 'typesense-3.example.net', 'port': '443', 'protocol': 'https' },
#  ]
# TYPESENSE_URL=ha://<redacted>

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>[email protected]</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003

####################################################################################
# Alternative API's External Address - Optional
#
# This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
# You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
# NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
# Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

Reproduction steps

1. Record video on Xpera 5 II with default camera
2. Sync via Immich
3. Try to watch it in Firefox

Additional information

I think the original video is encoded in h265/HEVC which isn't supported by Firefox. But I assume, since ffmpeg is configured to convert anything that isn't h264 to h264 that it should be converted correctly, which it apparently isn't.

Original video: Stream #0:10x2: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt2020nc/bt2020/arib-std-b67), 1920x1080, 18084 kb/s, 23.96 fps, 23.98 tbr, 90k tbn (default)

Video after encoding by immich: Stream #0:00x1: Video: h264 (High 10) (avc1 / 0x31637661), yuv420p10le(tv, bt2020nc/bt2020/arib-std-b67, progressive), 1280x720, 1070 kb/s, 23.98 fps, 23.98 tbr, 24k tbn (default)

Jul 02 '23 18:07 rastla

I have the same issue as well - but I see nothing in the logs that could suggest anything wrong happening.

In my case, playback in the app works, but playback does not work on the web at all. I have tested on Edge and Firefox, and playback in thumbnail and playback in viewer do not work. I can however verify that the video has been encoded, and playing back the file from /encoded-video is fine.

In case of Firefox, I am prompted with "No video with supported format and MIME type found".

In case of Edge, the player loading icon keeps looping with no notable network activity that would indicate loading.

In both cases, playback preview in thumbnail show the red ( ! ) icon. I have even tried spinning up a fresh instance of immich, but to no avail. This happens even on default ffmpeg settings.

My video example was shot on a GoPro in H265, and transcoded again into H265 to reduce storage use. This file was uploaded to immich, which would then transcode to H264. I unfortunately cannot share the file here as the contents are private, and I do not have the device to reproduce the behavior.

Original video:

Transcode by immich:

Jul 06 '23 10:07 ShinasShaji

The issue is because of the pixel format of the encoded videos. Firefox doesn't seem to support 10-bit H.264 video like yuv420p10le.

The good news is that fixing this just means adding a flag to convert to yuv420p. The bad news is that the video looks dull in Firefox, much more so than when played locally or in Chrome. I'd like to fix this, but I'm wondering if there's a way to make them look better in Firefox.

Jul 07 '23 06:07 mertalev

I see, but then I guess there's nothing to be done on the side of Immich about the issue. I also wanted to point out that this issue occurs in mobile Safari (iOS and iPadOS) as well, and can confirm that playback works in Chrome.

Thanks for the reply, I guess I'll just use Chrome then!

Jul 08 '23 17:07 ShinasShaji

The good news is that fixing this just means adding a flag to convert to yuv420p

As far as I can see currently there is no possibility to set that flag from the UI, right? Or do I set it somewhere in a config? I understand that the quality of the preview videos would be worse. But I'd rather have a worse looking video than no video at all.

Jul 19 '23 12:07 rastla

No, this is something that would be added to the ffmpeg command internally. I'm working on improving colors for HDR->SDR conversion, so this is something I'll be revisiting soon :)

Jul 19 '23 14:07 mertalev