metalangle icon indicating copy to clipboard operation
metalangle copied to clipboard

Published 20 hours ago • verified publisher icon kakashidinho

Add support for EGL_EXT_gl_colorspace_bt2020_pq

Open tmm1 opened this issue 5 years ago • 15 comments

WIP. Wired up the EGL_EXT_gl_colorspace_bt2020 extension, but have not tested it yet.

I'm not so sure about the public MGLKit API to expose this. I may decide to use eglCreateWindowSurface directly in my app and remove the MGLDrawableColorSpace/MGLDrawableColorFormat changes here.

tmm1 avatar Jun 29 '20 07:06 tmm1

BTW, I have been testing these changes and they basically work but somehow the colors on the screen were not correct compared to the reference AVPlayer.

After some investigation I discovered there is a bug in the translation of the fragment shader:

[libmpv_render] debug: fragment shader source:
...
[libmpv_render] debug: [ 43] // color mapping
[libmpv_render] debug: [ 44] color.rgb *= vec3(100.000000);
[libmpv_render] debug: [ 45] color.rgb *= vec3(0.010000);


[libmpv_render] debug: Translated shader:
...
[libmpv_render] debug: [ 83] (_ucolor.xyz *= vec3(100.0, 100.0, 100.0));
[libmpv_render] debug: [ 84] (_ucolor.xyz *= vec3(0.0099999998, 0.0099999998, 0.0099999998));

(translation comes from GetTranslatedShaderSourceANGLE)

Is this bug part of ANGLE or in SPIRV-cross?

tmm1 avatar Jun 30 '20 04:06 tmm1

That is translated shader source done by ANGLE. If you want to get translated metal shader done by spirv-cross, you can use glGetProgramBinaryOES, see example here: https://github.com/kakashidinho/metalangle/blob/78017f07e5adee0692506c94cdb299d92c854ad3/src/tests/gl_tests/ProgramBinaryTest.cpp#L94

I don't think the incorrect color is done by shader translation. Why do you think the above fragment shader translation is wrong? I only see the floating point number rounding errors (very small i.e 0.01 to 0.0099999998). The constructor calls are basically the same between original source and translated one. It might be caused by something else, maybe color space problem, etc.

kakashidinho avatar Jun 30 '20 04:06 kakashidinho

I'm not sure how is the bt2020 colorspace used to be honest? When you render a video frame to a texture, it is in linear space, isn't it? when the later step renders this texture onto the view layer in BT2020 colorspace, is there any special conversion step needs to be done?

kakashidinho avatar Jun 30 '20 04:06 kakashidinho

Another way to debug this type of mismatch rendering result is using glReadPixels to read an identical rendered frame's pixels into a float array, and then compare the array's content between reference player and MetalANGLE backed player.

kakashidinho avatar Jun 30 '20 04:06 kakashidinho

My understanding is not perfect, but here is how I believe it works:

The video image is in YUV format. Each plane is uploaded into separate 16bit texture. Then a shader is used to convert back to 10bit, combine the textures into one (via color matrix to convert YUV to RGB), and finally apply any color mapping transformations before sending to FBO. In the case of BT2020 PQ, no color map transformation is done (since the display handles it in transfer function, based on PQ equations).

My guess was floating point precision issue was causing changes to the luma and chroma planes affecting final colors slightly. But maybe you're right the effect is minimal and there is still some other problem.

[vf] v: [out] 3840x1608 yuv420p10 bt.2020-ncl/bt.2020/pq/limited/display SP=10.000000 CL=mpeg2/4/h264
[cplayer] info: VO: [libmpv] 3840x1608 yuv420p10
[cplayer] v: VO: Description: render API for libmpv
[vo/libmpv] v: reconfig to 3840x1608 yuv420p10 bt.2020-ncl/bt.2020/pq/limited/display SP=10.000000 CL=mpeg2/4/h264
[libmpv_render] v: Window size: 3840x2160
[libmpv_render] v: Video source: 3840x1608 (1:1)
[libmpv_render] v: Video display: (0, 0) 3840x1608 -> (0, 276) 3840x1608
[libmpv_render] v: Video scale: 1.000000/1.000000
[libmpv_render] v: OSD borders: l=0 t=276 r=0 b=276
[libmpv_render] v: Video borders: l=0 t=276 r=0 b=276
[libmpv_render] v: Texture for plane 0: 3840x1608
[libmpv_render] v: Texture for plane 1: 1920x804
[libmpv_render] v: Texture for plane 2: 1920x804
[libmpv_render] v: Testing FBO format rgba16f
[libmpv_render] debug: Resizing texture: 16x16
[libmpv_render] v: Using FBO format rgba16f.
[libmpv_render] v: No advanced processing required. Enabling dumb mode.
[libmpv_render] v: Testing FBO format rgba16f
[libmpv_render] debug: Resizing texture: 16x16
[libmpv_render] v: Using FBO format rgba16f.
[libmpv_render] v: Disabling HDR peak computation (one or more of the following is not supported: compute shaders=0, SSBO=0).
[libmpv_render] v: No advanced processing required. Enabling dumb mode.
[libmpv_render] debug: vertex shader source:
[libmpv_render] debug: [  1] #version 300 es
[libmpv_render] debug: [  2] #ifdef GL_FRAGMENT_PRECISION_HIGH
[libmpv_render] debug: [  3] precision highp float;
[libmpv_render] debug: [  4] #else
[libmpv_render] debug: [  5] precision mediump float;
[libmpv_render] debug: [  6] #endif
[libmpv_render] debug: [  7] precision mediump sampler2D;
[libmpv_render] debug: [  8] precision mediump sampler3D;
[libmpv_render] debug: [  9] #define tex1D texture
[libmpv_render] debug: [ 10] #define tex3D texture
[libmpv_render] debug: [ 11] #define LUT_POS(x, lut_size) mix(0.5 / (lut_size), 1.0 - 0.5 / (lut_size), (x))
[libmpv_render] debug: [ 12] in vec2 vertex_position;
[libmpv_render] debug: [ 13] in vec2 vertex_texcoord0;
[libmpv_render] debug: [ 14] out vec2 texcoord0;
[libmpv_render] debug: [ 15] in vec2 vertex_texcoord1;
[libmpv_render] debug: [ 16] out vec2 texcoord1;
[libmpv_render] debug: [ 17] in vec2 vertex_texcoord2;
[libmpv_render] debug: [ 18] out vec2 texcoord2;
[libmpv_render] debug: [ 19] void main() {
[libmpv_render] debug: [ 20] gl_Position = vec4(vertex_position, 1.0, 1.0);
[libmpv_render] debug: [ 21] texcoord0 = vertex_texcoord0;
[libmpv_render] debug: [ 22] texcoord1 = vertex_texcoord1;
[libmpv_render] debug: [ 23] texcoord2 = vertex_texcoord2;
[libmpv_render] debug: [ 24] }
[libmpv_render] debug: Translated shader:
[libmpv_render] debug: [  1] #version 450 core
[libmpv_render] debug: [  2] @@ XFB-DECL @@
[libmpv_render] debug: [  3]
[libmpv_render] debug: [  4] layout (constant_id=0) const bool ANGLERasterizationDisabled = false;
[libmpv_render] debug: [  5] struct ANGLEDepthRangeParams {
[libmpv_render] debug: [  6] float near;
[libmpv_render] debug: [  7] float far;
[libmpv_render] debug: [  8] float diff;
[libmpv_render] debug: [  9] float reserved;
[libmpv_render] debug: [ 10] };
[libmpv_render] debug: [ 11] @@ LAYOUT-vertex_position() @@@@ QUALIFIER-vertex_position() @@ vec2 _uvertex_position;
[libmpv_render] debug: [ 12] @@ LAYOUT-vertex_texcoord0() @@@@ QUALIFIER-vertex_texcoord0() @@ vec2 _uvertex_texcoord0;
[libmpv_render] debug: [ 13] @@ LAYOUT-texcoord0() @@@@ QUALIFIER-texcoord0() @@ vec2 _utexcoord0;
[libmpv_render] debug: [ 14] @@ LAYOUT-vertex_texcoord1() @@@@ QUALIFIER-vertex_texcoord1() @@ vec2 _uvertex_texcoord1;
[libmpv_render] debug: [ 15] @@ LAYOUT-texcoord1() @@@@ QUALIFIER-texcoord1() @@ vec2 _utexcoord1;
[libmpv_render] debug: [ 16] @@ LAYOUT-vertex_texcoord2() @@@@ QUALIFIER-vertex_texcoord2() @@ vec2 _uvertex_texcoord2;
[libmpv_render] debug: [ 17] @@ LAYOUT-texcoord2() @@@@ QUALIFIER-texcoord2() @@ vec2 _utexcoord2;
[libmpv_render] debug: [ 18] @@ LAYOUT-ANGLEUniformBlock() @@@@ QUALIFIER-ANGLEUniformBlock() @@ ANGLEUniformBlock{
[libmpv_render] debug: [ 19] vec4 viewport;
[libmpv_render] debug: [ 20] float halfRenderAreaHeight;
[libmpv_render] debug: [ 21] float viewportYScale;
[libmpv_render] debug: [ 22] float negViewportYScale;
[libmpv_render] debug: [ 23] uint xfbActiveUnpaused;
[libmpv_render] debug: [ 24] ivec4 xfbBufferOffsets;
[libmpv_render] debug: [ 25] uvec4 acbBufferOffsets;
[libmpv_render] debug: [ 26] ANGLEDepthRangeParams depthRange;
[libmpv_render] debug: [ 27] uint clipDistancesEnabled;
[libmpv_render] debug: [ 28] uint coverageMask;
[libmpv_render] debug: [ 29] } ANGLEUniforms;
[libmpv_render] debug: [ 30]
[libmpv_render] debug: [ 31] #ifdef ANGLE_ENABLE_LINE_SEGMENT_RASTERIZATION
[libmpv_render] debug: [ 32] @@ LAYOUT-ANGLEPosition() @@@@ QUALIFIER-ANGLEPosition() @@ vec4 ANGLEPosition;
[libmpv_render] debug: [ 33]
[libmpv_render] debug: [ 34] #endif
[libmpv_render] debug: [ 35] void main(){
[libmpv_render] debug: [ 36] (gl_Position = vec4(_uvertex_position, 1.0, 1.0));
[libmpv_render] debug: [ 37] (_utexcoord0 = _uvertex_texcoord0);
[libmpv_render] debug: [ 38] (_utexcoord1 = _uvertex_texcoord1);
[libmpv_render] debug: [ 39] (_utexcoord2 = _uvertex_texcoord2);
[libmpv_render] debug: [ 40]
[libmpv_render] debug: [ 41] #ifdef ANGLE_ENABLE_LINE_SEGMENT_RASTERIZATION
[libmpv_render] debug: [ 42] (ANGLEPosition = gl_Position);
[libmpv_render] debug: [ 43]
[libmpv_render] debug: [ 44] #endif
[libmpv_render] debug: [ 45] @@ XFB-OUT @@;
[libmpv_render] debug: [ 46] (gl_Position.z = (gl_Position.z * ANGLEUniforms.depthRange.reserved));
[libmpv_render] debug: [ 47] (gl_Position.z = ((gl_Position.z + gl_Position.w) * 0.5));
[libmpv_render] debug: [ 48] (gl_Position.y = (gl_Position.y * ANGLEUniforms.negViewportYScale));
[libmpv_render] debug: [ 49] if (ANGLERasterizationDisabled)
[libmpv_render] debug: [ 50] {
[libmpv_render] debug: [ 51] (gl_Position = vec4(-3.0, -3.0, -3.0, 1.0));
[libmpv_render] debug: [ 52] }
[libmpv_render] debug: [ 53] }
[libmpv_render] debug: fragment shader source:
[libmpv_render] debug: [  1] #version 300 es
[libmpv_render] debug: [  2] #ifdef GL_FRAGMENT_PRECISION_HIGH
[libmpv_render] debug: [  3] precision highp float;
[libmpv_render] debug: [  4] #else
[libmpv_render] debug: [  5] precision mediump float;
[libmpv_render] debug: [  6] #endif
[libmpv_render] debug: [  7] precision mediump sampler2D;
[libmpv_render] debug: [  8] precision mediump sampler3D;
[libmpv_render] debug: [  9] #define tex1D texture
[libmpv_render] debug: [ 10] #define tex3D texture
[libmpv_render] debug: [ 11] #define LUT_POS(x, lut_size) mix(0.5 / (lut_size), 1.0 - 0.5 / (lut_size), (x))
[libmpv_render] debug: [ 12] out vec4 out_color;
[libmpv_render] debug: [ 13] in vec2 texcoord0;
[libmpv_render] debug: [ 14] in vec2 texcoord1;
[libmpv_render] debug: [ 15] in vec2 texcoord2;
[libmpv_render] debug: [ 16] uniform mat3 colormatrix;
[libmpv_render] debug: [ 17] uniform vec3 colormatrix_c;
[libmpv_render] debug: [ 18] uniform vec3 src_luma;
[libmpv_render] debug: [ 19] uniform vec3 dst_luma;
[libmpv_render] debug: [ 20] uniform sampler2D texture0;
[libmpv_render] debug: [ 21] uniform vec2 texture_size0;
[libmpv_render] debug: [ 22] uniform mat2 texture_rot0;
[libmpv_render] debug: [ 23] uniform vec2 texture_off0;
[libmpv_render] debug: [ 24] uniform vec2 pixel_size0;
[libmpv_render] debug: [ 25] uniform sampler2D texture1;
[libmpv_render] debug: [ 26] uniform vec2 texture_size1;
[libmpv_render] debug: [ 27] uniform mat2 texture_rot1;
[libmpv_render] debug: [ 28] uniform vec2 texture_off1;
[libmpv_render] debug: [ 29] uniform vec2 pixel_size1;
[libmpv_render] debug: [ 30] uniform sampler2D texture2;
[libmpv_render] debug: [ 31] uniform vec2 texture_size2;
[libmpv_render] debug: [ 32] uniform mat2 texture_rot2;
[libmpv_render] debug: [ 33] uniform vec2 texture_off2;
[libmpv_render] debug: [ 34] uniform vec2 pixel_size2;
[libmpv_render] debug: [ 35] void main() {
[libmpv_render] debug: [ 36] vec4 color = vec4(0.0, 0.0, 0.0, 1.0);
[libmpv_render] debug: [ 37] color.r = 64.250000 * vec4(texture(texture0, texcoord0)).r;
[libmpv_render] debug: [ 38] color.g = 64.250000 * vec4(texture(texture1, texcoord1)).r;
[libmpv_render] debug: [ 39] color.b = 64.250000 * vec4(texture(texture2, texcoord2)).r;
[libmpv_render] debug: [ 40] color = color.rgbr;
[libmpv_render] debug: [ 41] color.rgb = mat3(colormatrix) * color.rgb + colormatrix_c;
[libmpv_render] debug: [ 42] color.a = 1.0;
[libmpv_render] debug: [ 43] // color mapping
[libmpv_render] debug: [ 44] color.rgb *= vec3(100.000000);
[libmpv_render] debug: [ 45] color.rgb *= vec3(0.010000);
[libmpv_render] debug: [ 46] out_color = color;
[libmpv_render] debug: [ 47] }
[libmpv_render] debug: Translated shader:
[libmpv_render] debug: [  1] #version 450 core
[libmpv_render] debug: [  2]
[libmpv_render] debug: [  3] @@ LAYOUT-defaultUniforms(std140) @@ uniform defaultUniforms
[libmpv_render] debug: [  4] {
[libmpv_render] debug: [  5]     mat3 _ucolormatrix;
[libmpv_render] debug: [  6]     vec3 _ucolormatrix_c;
[libmpv_render] debug: [  7]     vec3 _usrc_luma;
[libmpv_render] debug: [  8]     vec3 _udst_luma;
[libmpv_render] debug: [  9]     vec2 _utexture_size0;
[libmpv_render] debug: [ 10]     mat2 _utexture_rot0;
[libmpv_render] debug: [ 11]     vec2 _utexture_off0;
[libmpv_render] debug: [ 12]     vec2 _upixel_size0;
[libmpv_render] debug: [ 13]     vec2 _utexture_size1;
[libmpv_render] debug: [ 14]     mat2 _utexture_rot1;
[libmpv_render] debug: [ 15]     vec2 _utexture_off1;
[libmpv_render] debug: [ 16]     vec2 _upixel_size1;
[libmpv_render] debug: [ 17]     vec2 _utexture_size2;
[libmpv_render] debug: [ 18]     mat2 _utexture_rot2;
[libmpv_render] debug: [ 19]     vec2 _utexture_off2;
[libmpv_render] debug: [ 20]     vec2 _upixel_size2;
[libmpv_render] debug: [ 21] };
[libmpv_render] debug: [ 22] layout (constant_id=0) const bool ANGLECoverageMaskEnabled = false;
[libmpv_render] debug: [ 23] void ANGLEWriteSampleMask(uint mask)
[libmpv_render] debug: [ 24] {
[libmpv_render] debug: [ 25]    if (ANGLECoverageMaskEnabled)
[libmpv_render] debug: [ 26]    {
[libmpv_render] debug: [ 27]        gl_SampleMask[0] = int(mask);
[libmpv_render] debug: [ 28]    }
[libmpv_render] debug: [ 29] }
[libmpv_render] debug: [ 30] vec4 flippedFragCoord;
[libmpv_render] debug: [ 31] struct ANGLEDepthRangeParams {
[libmpv_render] debug: [ 32] float near;
[libmpv_render] debug: [ 33] float far;
[libmpv_render] debug: [ 34] float diff;
[libmpv_render] debug: [ 35] float reserved;
[libmpv_render] debug: [ 36] };
[libmpv_render] debug: [ 37] @@ LAYOUT-out_color() @@out vec4 _uout_color;
[libmpv_render] debug: [ 38] @@ LAYOUT-texcoord0() @@@@ QUALIFIER-texcoord0() @@ vec2 _utexcoord0;
[libmpv_render] debug: [ 39] @@ LAYOUT-texcoord1() @@@@ QUALIFIER-texcoord1() @@ vec2 _utexcoord1;
[libmpv_render] debug: [ 40] @@ LAYOUT-texcoord2() @@@@ QUALIFIER-texcoord2() @@ vec2 _utexcoord2;
[libmpv_render] debug: [ 41] @@ LAYOUT-texture0() @@@@ QUALIFIER-texture0() @@ sampler2D _utexture0;
[libmpv_render] debug: [ 42] @@ LAYOUT-texture1() @@@@ QUALIFIER-texture1() @@ sampler2D _utexture1;
[libmpv_render] debug: [ 43] @@ LAYOUT-texture2() @@@@ QUALIFIER-texture2() @@ sampler2D _utexture2;
[libmpv_render] debug: [ 44] @@ LAYOUT-ANGLEUniformBlock() @@@@ QUALIFIER-ANGLEUniformBlock() @@ ANGLEUniformBlock{
[libmpv_render] debug: [ 45] vec4 viewport;
[libmpv_render] debug: [ 46] float halfRenderAreaHeight;
[libmpv_render] debug: [ 47] float viewportYScale;
[libmpv_render] debug: [ 48] float negViewportYScale;
[libmpv_render] debug: [ 49] uint xfbActiveUnpaused;
[libmpv_render] debug: [ 50] ivec4 xfbBufferOffsets;
[libmpv_render] debug: [ 51] uvec4 acbBufferOffsets;
[libmpv_render] debug: [ 52] ANGLEDepthRangeParams depthRange;
[libmpv_render] debug: [ 53] uint clipDistancesEnabled;
[libmpv_render] debug: [ 54] uint coverageMask;
[libmpv_render] debug: [ 55] } ANGLEUniforms;
[libmpv_render] debug: [ 56]
[libmpv_render] debug: [ 57] #ifdef ANGLE_ENABLE_LINE_SEGMENT_RASTERIZATION
[libmpv_render] debug: [ 58] @@ LAYOUT-ANGLEPosition() @@@@ QUALIFIER-ANGLEPosition() @@ vec4 ANGLEPosition;
[libmpv_render] debug: [ 59]
[libmpv_render] debug: [ 60] #endif
[libmpv_render] debug: [ 61] void main(){
[libmpv_render] debug: [ 62]
[libmpv_render] debug: [ 63] #ifdef ANGLE_ENABLE_LINE_SEGMENT_RASTERIZATION
[libmpv_render] debug: [ 64] flippedFragCoord = vec4(gl_FragCoord);
[libmpv_render] debug: [ 65] (flippedFragCoord.y = (((gl_FragCoord.y - ANGLEUniforms.halfRenderAreaHeight) * ANGLEUniforms.viewportYScale) + ANGLEUniforms.halfRenderAreaHeight));
[libmpv_render] debug: [ 66] vec2 s91c = (((((ANGLEPosition.xy / ANGLEPosition.w) * 0.5) + 0.5) * ANGLEUniforms.viewport.zw) + ANGLEUniforms.viewport.xy);
[libmpv_render] debug: [ 67] vec2 s91d = abs((s91c - flippedFragCoord.xy));
[libmpv_render] debug: [ 68] vec2 s91e = ((s91d * s91d) * 2.0);
[libmpv_render] debug: [ 69] vec2 s91f = ((s91e + s91e.yx) - s91d);
[libmpv_render] debug: [ 70] if (((s91f.x > 9.9999997e-06) && (s91f.y > 9.9999997e-06)))
[libmpv_render] debug: [ 71] {
[libmpv_render] debug: [ 72] discard;
[libmpv_render] debug: [ 73] }
[libmpv_render] debug: [ 74]
[libmpv_render] debug: [ 75] #endif
[libmpv_render] debug: [ 76] vec4 _ucolor = vec4(0.0, 0.0, 0.0, 1.0);
[libmpv_render] debug: [ 77] (_ucolor.x = (64.25 * vec4(texture(_utexture0, _utexcoord0)).x));
[libmpv_render] debug: [ 78] (_ucolor.y = (64.25 * vec4(texture(_utexture1, _utexcoord1)).x));
[libmpv_render] debug: [ 79] (_ucolor.z = (64.25 * vec4(texture(_utexture2, _utexcoord2)).x));
[libmpv_render] debug: [ 80] (_ucolor = _ucolor.xyzx);
[libmpv_render] debug: [ 81] (_ucolor.xyz = ((mat3(_ucolormatrix) * _ucolor.xyz) + _ucolormatrix_c));
[libmpv_render] debug: [ 82] (_ucolor.w = 1.0);
[libmpv_render] debug: [ 83] (_ucolor.xyz *= vec3(100.0, 100.0, 100.0));
[libmpv_render] debug: [ 84] (_ucolor.xyz *= vec3(0.0099999998, 0.0099999998, 0.0099999998));
[libmpv_render] debug: [ 85] (_uout_color = _ucolor);
[libmpv_render] debug: [ 86] if (ANGLECoverageMaskEnabled)
[libmpv_render] debug: [ 87] {
[libmpv_render] debug: [ 88] ANGLEWriteSampleMask(ANGLEUniforms.coverageMask);
[libmpv_render] debug: [ 89] }
[libmpv_render] debug: [ 90] }
[libmpv_render] debug: shader link log (status=1):
[cplayer] v: first video frame after restart shown

tmm1 avatar Jun 30 '20 07:06 tmm1

If you want to get translated metal shader done by spirv-cross, you can use glGetProgramBinaryOES

This is very helpful for debugging, thank you! I will try it tomorrow and see if the metal shader source looks how I expect.

tmm1 avatar Jun 30 '20 07:06 tmm1

To check quickly, I added a breakpoint in ProgramMtl::createMslShader to look at the metal shader. It has a similar floating point error:

    float3 _112 = _ucolor.xyz * float3(100.0);
    _ucolor = float4(_112.x, _112.y, _112.z, _ucolor.w);
    float3 _119 = _ucolor.xyz * float3(0.00999999977648258209228515625);
    _ucolor = float4(_119.x, _119.y, _119.z, _ucolor.w);
    out._uout_color = _ucolor;

Then I removed these lines from the original GLSL shader, and the colors are still not correct. So you were right that this is a red herring.

I also tried MoltenVK with libmpv (using vulkan mode instead of GL), and the colors are exactly the same as with MetalANGLE. So maybe it is an OS issue. I plan to test on more OS/device combinations to narrow down the bug.

tmm1 avatar Jul 01 '20 00:07 tmm1

I’m curious anw, why multiply by 100 then divide by 100 (multiply by 0.01). Doesn’t it produce the (nearly) exact same old value? Maybe I misunderstood something.

kakashidinho avatar Jul 01 '20 00:07 kakashidinho

The shader is generated programmatically. In this case there's nothing in-between, because no color mapping is being applied.

https://github.com/mpv-player/mpv/blob/dc24a437fb9ba2485bc8c5866542bd36254410d4/video/out/gpu/video_shaders.c#L827

tmm1 avatar Jul 01 '20 00:07 tmm1

I just noticed that kCGColorSpaceITUR_2020_PQ is deprecated https://developer.apple.com/documentation/coregraphics/kcgcolorspaceitur_2020_pq?language=objc

kakashidinho avatar Jul 01 '20 06:07 kakashidinho 

CG_EXTERN const CFStringRef kCGColorSpaceITUR_2020_PQ_EOTF
CG_AVAILABLE_BUT_DEPRECATED(10.14.6, 10.15.4, 12.6, 13.4); // Use kCGColorSpaceITUR_2100_PQ. This is only a name change, color space remains the same

CG_EXTERN const CFStringRef kCGColorSpaceITUR_2020_PQ
CG_AVAILABLE_BUT_DEPRECATED(10.15.4, 10.16, 13.4, 14.0);   // Use kCGColorSpaceITUR_2100_PQ. This is only a name change, color space remains the same

CG_EXTERN const CFStringRef kCGColorSpaceITUR_2100_PQ
CG_AVAILABLE_STARTING(10.16, 14.0);

tmm1 avatar Jul 01 '20 16:07 tmm1

This appears not to work correctly on Apple TV due to tvOS bugs. I have filed a report with Apple:

FB7841133 CAMetalLayer with pixelFormat=MTLPixelFormatRGBA16Float and colorspace=kCGColorSpaceITUR_2020_PQ_EOTF will not render HDR colors correctly On tvOS 13, using kCGColorSpaceITUR_2020_PQ_EOTF colorspace on a CAMetalLayer results in only black screen on Apple TV 4K with HDR display attached. On tvOS 14 Beta, the CAMetalLayer will render its contents to the screen and applies the PQ function but the resulting colors are much brighter than expected.

tmm1 avatar Jul 03 '20 15:07 tmm1

ok I see, thanks. Do you have small sample code to demonstrate the bug? Apple tend to ask for code to reproduce the bug, using MetalANGLE to reproduce could be complicated. They are quite slow to respond also.

kakashidinho avatar Jul 04 '20 03:07 kakashidinho

It seems to me that it is simply unsupported at this time outside macOS. But given that the first tvOS 14 Beta makes some things render now, I am hopeful they are working on improving support.

If you look at the macOS docs, there are several extra methods provided to allow color-correct rendering. See:

  • https://developer.apple.com/documentation/metal/drawable_objects/displaying_hdr_content_in_a_metal_layer/using_color_spaces_to_display_hdr_content?language=objc
  • https://developer.apple.com/documentation/metal/drawable_objects/displaying_hdr_content_in_a_metal_layer/using_system_tone_mapping_on_video_content?language=objc

Hopefully these will be copied over to tvOS and iOS soon:

  • metalLayer has wantsExtendedDynamicRangeContent property
  • metalLayer supports MTLPixelFormatBGR10A2Unorm format
  • metalLayer has EDRMetadata property to set HDR10 and HLG tone-mapping properties

tmm1 avatar Jul 06 '20 16:07 tmm1

As far as a repro, probably closest available right now is @qiudaomao's MoltenVK example in https://github.com/qiudaomao/MPVColorIssue

One thing I found interesting: with the MoltenVK example, it renders colors correctly inside the Apple TV simulator on macOS. But when I tried the same thing with MetalANGLE and this branch, I only got a black screen in the Apple TV simulator.

tmm1 avatar Jul 06 '20 16:07 tmm1