Color Shift Between Quicktime and AV Pro for Android

[avclubvids]

There is a noticeable color shift between an h265 played in Quicktime or other video players on OSX and Unity/AVP/Android. It is not a simple gamma shift, as hue and saturation are affected too. Reference images:

h265 in Quicktime (correct):

h265 in Unity using AVP (incorrect, looks like this on a Quest as well):

h265 in Unity using the Unity Video Player (pretty much correct):

h265 in Unity using AVP + manual color correction with Post Processing Stack v2 (subjective but acceptable):

h265 converted to h264 using ffmpeg to change color metadata, in Unity using AVP with no post-processing (close to correct):

• Unity version: 2021.3.8.f1
• AVPro Video version: Core 2.7.3
• Operating system version: OSX 13.1
• Device model: MBP 16-inch, 2019
• Video specs (resolution, frame-rate, codec, file size): 4096x4096, 29.97fps, h265, 8.68GB

To Reproduce

1. load an h265 in Unity load into a MediaPlayer, and observe color shift by comparing it with the same file open in a video player like Quicktime

My thought is that the color space flags in the files are either being read incorrectly or there needs to be a user-editable setting for what AVP should assume a video is in terms of gamma and gamut. The above h264 is close enough to be acceptable, I made it using the advice in this thread: https://github.com/RenderHeads/UnityPlugin-AVProMovieCapture/issues/55.

The h265 shows this in ffprobe:

Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 110595 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

And the h264 I made from it is:

Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 12303 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

[MorrisRH]

So both Android and macOS editor are producing the same output, but this differs from the output as seen in Quicktime?

I'm a little confused as the ffprobe output you've pasted at the end is showing the codec as being h264 in both cases, although the first one is using the full colour range (yuvj420p).

Are you able to share your test video? If so please send a link to it to [email protected] mentioning this issue in the emails subject. It will certainly help in trying to diagnose what is happening.

[avclubvids]

Sorry I gave the wrong ffprobe data for the h265 (...too many test videos...). I'm emailing you 4 different files I just made for testing, I'm stepping through them now in Unity to see what is what. Here's the ffprobe for them:

4k_testPatterns_sRGB_g22_h264_video_v001.mov: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 3658 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

4k_testPatterns_sRGB_g22_h264_full_v001.mov: Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 3892 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

4k_testPatterns_sRGB_g22_h265_full_v001.mov: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(pc, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 993 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

4k_testPatterns_sRGB_g22_h265_video_v001.mov: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 894 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

[avclubvids]

@MorrisRH - to answer your question:

So both Android and macOS editor are producing the same output, but this differs from the output as seen in Quicktime?

Yes, that is correct. Full-range movie files appear the same in Resolve/Quicktime/etc. but they have a large shift when viewed in Unity's editor using AVP, as well as in the headset (Quest 2) with a built app.

[avclubvids]

Ok, screenshot comparisons of the above 4 video files are below. In each screencap Unity/AVP are on the left and Quicktime is on the right. As you can see there is an issue where videos with Full range pixel values ("pc" flag in ffprobe) are definitely not read correctly in AVP/Unity. Even the Video range ("tv" flag in ffprobe) files are not totally correct - there is a slight red hue to everything in AVP. Worth noting: h264 and h265 have the same color throughout so this is not a codec issue per se:

h264 Full Range:

h264 Video range:

h265 Full range:

h265 Video range:

[MorrisRH]

Sorry I gave the wrong ffprobe data for the h265 (...too many test videos...). I'm emailing you 4 different files I just made for testing, I'm stepping through them now in Unity to see what is what. Here's the ffprobe for them:

4k_testPatterns_sRGB_g22_h264_video_v001.mov: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 3658 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

4k_testPatterns_sRGB_g22_h264_full_v001.mov: Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 3892 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

4k_testPatterns_sRGB_g22_h265_full_v001.mov: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(pc, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 993 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

4k_testPatterns_sRGB_g22_h265_video_v001.mov: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt709/bt709/unknown, progressive), 4096x4096 [SAR 1:1 DAR 1:1], 894 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)

Still waiting for these to arrive, but that'll be really helpful - thanks.

[avclubvids]

I emailed a Dropbox link to [email protected] - did it possibly get stuck in a spam folder?

[Chris-RH]

We've not received it. Nothing in the spam folder either. How long ago did you send it? Could you double check the email address and resend please?

[avclubvids]

It's definitely sent to the email address provided... here, there's nothing sensitive about this link so please see if this works:

https://www.dropbox.com/sh/lfthpjk2r32z86s/AAAZlDZ3E-4am58Xde5i8IOta?dl=0

[MorrisRH]

Thanks for the link. I've downloaded the videos and will investigate.

[avclubvids]

@MorrisRH I ran out of room in my Dropbox so I had to move those videos out, let me know if you still need them and I'll put them back and send the new generated link

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[avclubvids]

@MorrisRH @Chris-RH – are you guys seeing this color shift on your end?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[avclubvids]

Was this issue ever able to be resolved?

[Chris-RH]

Its still being looked into

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Chris-RH]

Can you see if there has been any improvement with the latest release (2.8.1) please? :)

[avclubvids]

I'm testing with more media and settings now, but initially, it looks like the reddish tint is gone. Is there anything in particular that I should A/B in the new version with the old?

[acgourley]

I've chased my tail thinking QuickTime was ground truth before. I then found out Apple uses a nonstandard gamma space. https://larryjordan.com/articles/caution-premiere-pro-final-cut-pro-x-and-quicktime-player-dont-show-the-same-color-the-same-way/

Try opening the video in VLC or ElMedia Player which I've found behave like every other video player on the planet except QuickTime.

[avclubvids]

In this case the ground truth is Resolve, which I have a 1:1 color match with in Quicktime, then once it hits AVP in Unity the color shift appears. The color matches in Resolve, QT, VLC, DJV_View, and CineSync.

[acgourley]

I see. Actually I am seeing something similar, AVPro not reading yuvj420p (PC range) files correctly. For example, those captured from a GoPro. I took a video of a color calibration card, and loaded it in the AVPro demo to compare it to the Quicktime rendition.

I believe AVPro's reading of the file is not correctly converting from the PC color range to the TV color range. As a result, it's showing darks darker, and lights lighter. I made a screenshot below with my findings (the luminance value is of the 3rd grey from the left). You can see my test file here.

Note: other viewers agree with Quicktime, so it's not just the Quicktime gamma curve selection at play here.

[Chris-RH]

Yes, that is quite a clear difference. Interesting that such a small difference in luminance value can make a big difference,

[MorrisRH]

Thanks for the test media. There is definitely an issue with 10bit full range videos not being handled correctly. We will get this fixed for the next release

[acgourley]

I sent the 10-bit file by mistake in my previous post. The screenshot above is of the 8-bit file. I just bring that up to indicate I believe it's a PC range issue, not a 10-bit issue.

8 Bit Test Clip 10 Bit Test Clip

I haven't actually done the test on the 10-bit one, so I hope having both in your hands will be helpful for confirmation.

p.s. you guys are awesome thanks for always being on top of things.

[MorrisRH]

I've fixed up macOS and iOS so that full range is now handled correctly and the fix will make it into the next release.

[avclubvids]

will the fix also work in Android? that's what started all these troubles... (Quest 2)

[Ste-RH]

Can you give us a short update on the issue with Android @acgourley ?

• Do you have a video that shows it off (maybe we already have it?) and is it encoded as full colour range?
• Are you in Linear or Gamma colour space?
• Are you using OES or none-OES?
• What shader are you using?
• Have you tried other devices than the Quest 2?

[acgourley]

I never tried on android, that was @avclubvids

[Ste-RH]

Apologies, picked the wrong person from the auto list :)

[acgourley]

I've fixed up macOS and iOS so that full range is now handled correctly and the fix will make it into the next release.

I wanted to confirm 2.8.5 seems to be working for me on OSX, with the displayed color very close to quicktimes for the same video. For anyone testing this, make sure to restart Unity after updating AVPro.