HDR videos look pale on mobile

[s0nerik]

The bug

Looks like video_player package doesn’t support HDR colors in videos. The respective issue comments seem to suggest that an alternative package, called media_kit, exists, which doesn’t have this problem.

The OS that Immich Server is running on

Raspbian

Version of Immich Server

v1.86.0

Version of Immich Mobile App

v1.86.0 build.126

Platform with the issue

• [ ] Server
• [ ] Web
• [X] Mobile

Your docker-compose.yml content

-

Your .env content

-

Reproduction steps

1. Shoot an HDR video on any iPhone supporting it.
2. Sync it in original quality into Immich.
3. Open synced video in the Immich app on that same iPhone.
4. Colors in the video look pale.

Additional information

No response

[mertalev]

We've been looking into media_kit as a better alternative for HDR and hope to switch to it. However, the impression we got was that while its color reproduction is definitely better, it still isn't HDR. Based on this patch in their repo, it seems there's a hardware decoding limitation involved. HDR seems to be an open issue in the Flutter ecosystem, unfortunately.

[s0nerik]

I think another alternative could be to use the native view for a video player on iOS, without copying frames into a texture. I guess native_video_player does that, but not sure as I haven’t tried it.

[mertalev]

I think platform views with Metal are the only way to get HDR on iOS. media_kit describes its player as native too, so there seems to be more to it than that. I'm not sure what this player uses.

[alexmercerind]

I saw this discussion.

Based on this patch in their repo, it seems there's a hardware decoding limitation involved.

That patch addresses a regression. 4K HDR plays flawless (performance wise) on iOS based on the feedback we have so far.

We have discussed few things in a new ticket:

https://github.com/media-kit/media-kit/issues/615

You can improve overall rendering result on normal displays with tone mapping.

I believe default one isn't best on iOS/macOS right now.

[mertalev]

Thank you for clarifying that! I think the --tone-mapping flag can help as well here, but we already transcode and tone-map videos, so the problem only occurs when users want to view the original. Does this flag have any color or performance impact on SDR videos, or is it safe to enable universally?

I also wonder if there's any hope for hardware-decoded HDR using Flutter. From the discussion you linked, it seems the limitations are on Flutter textures along with libmpv. Am I right in thinking that platform views with Metal would allow for HDR? If so, are you aware of any player that takes this approach?

[yotis1982]

Sorry if my knowledge is not at best to fully understand this issue, but is this topic being addressed on the roadmap or at least a fix expected in the future releases? Is a bit weird to look at my videos on iPhone and look like they are washed out.

Thank you!

[mertalev]

Do you have transcoding disabled? HDR videos have to be transcoded for the colors to look normal on mobile.

We do have a PoC with media_kit that can render these videos better (albeit not HDR), but it has some other issues we haven't fixed yet.

[yotis1982]

@mertalev thanks for fast reply, transcoding settings are pretty much the default ones; for the Trandcode Policy I have "Only videos not in desired format". Should I have it on "All videos" to transcode everything?

Thanks!

[mertalev]

Hmm, it should be transcoding on default settings. Could you try running a "missing" job for transcoding? If that doesn't transcode anything then it might just be a bug.

[yotis1982]

I tried your suggestion, the task went fast, I don't think anything was transcoded. I'll check more on. Thanks!

[yotis1982]

I did a test today and it seems to work fine (aka colors looks fine in the HDR), but is interesting how it works at least for me. I'll do my best to explain and maybe somebody with more technical knowledge in this area will find it useful.

First the Immich server Settings for Transcode Policy is set to "Only videos not in desired format"; everything else is default as per installation.

I took a video on my iPhone and played the video through the Immich app.

1. Video is only local on iPhone (backup icon is cloud cut) - colors are faded (see below for details):

The Info shows "is remote: false" and "storage : AssetState.local"

2. Video is local on iPhone and backup to Immich (backup icon is cloud and checkmark) - colors are faded (see below for details)

The Info shows "is remote: true" and "storage : AssetState.merged".

3. Video is only backup on Immich (backup icon is just cloud) and deleted from Iphone - colors are normal (see below for details).

The info shows "is remote: false" and "storage : AssetState.remote"

I think everything is fine, because my only concern was when I was about to check on backup videos that the colors will not look ok. If the video is still on my iPhone, I can play it via the native iOS app.

Thanks and sorry for the large images.

[benbeton]

thank you for the testing an Details. Nevertheless in my opinion the colors should never be shown faded, other users may ask themselves the same questions and maybe worried about. But I think It's really helpful to have your analysis

[aureateflux]

Is this only an IOS issue? I'm seeing this on my android app as well. I have a video that is marked "10-bit HDR" in Google Photos that looks washed out in the Immich app on Android, but colors are normal on the Immich web view (the web view doesn't appear to be HDR, but the colors look similar to the native video).

[baldemar-wuda]

Same here (iOS) - HDR video colors are "off" in the app, but fine when played via web browser.

[s0nerik]

Is this only an IOS issue?

I'm seeing the same issue on Android as well with HDR videos recorded by iPhone 13 mini.

[mertalev]

I believe the issue affects both iOS and Android. The underlying native players for both platforms support HDR, but the way Flutter interacts with them doesn't allow for it.

[blue-kaleidoscope]

Using Immich on iPadOS HDR looks good and not pale. 🤔

[s0nerik]

Using Immich on iPadOS HDR looks good and not pale. 🤔

You're probably viewing the processed videos (where HDR was converted to SDR). The problem is with viewing the original HDR video.

[blue-kaleidoscope]

That is possible, as there was some time difference between watching an HDR video on my iPhone, and then on my iPad.