Sunshine Sunshine incorrectly encodes using the BT.601 colourspace instead of BT.709, resulting in very incorrect colour reproduction on any connected client.

Describe the Bug

Sunshine incorrectly encodes using the BT.601 colourspace instead of BT.709, resulting in very incorrect colour reproduction on any connected client.

This behaviour does not occur with either Parsec (which I most often use) or NVIDIA GeForce Experience.

Sunshine log output also confirms the issue.

[2022:08:06:04:34:50]: Info:
Device Description : NVIDIA GeForce RTX 2060
Device Vendor ID   : 0x000010DE
Device Device ID   : 0x00001F08
Device Video Mem   : 5980 MiB
Device Sys Mem     : 0 MiB
Share Sys Mem      : 8111 MiB
Feature Level      : 0x0000B100
Capture size       : 1920x1080
Offset             : 0x0
Virtual Desktop    : 1920x1080
[2022:08:06:04:34:50]: Info: Color coding [Rec. 601]
[2022:08:06:04:34:50]: Info: Color range: [MPEG]

Screenshots

Open the Parsec/NVIDIA GFE/Sunshine screenshots in separate tabs and switch between them to see the difference.

Parsec (correct, BT.709)

Moonlight + NVIDIA GeForce Experience (correct, BT.709)

Moonlight + Sunshine (incorrect, BT.601)

Sunshine Host Operating System and Version

Windows 10 19044.1865 21H2

Architecture

x86_64

Sunshine Version

0.14.0

GPU

NVIDIA RTX 2060

GPU Driver Version

516.59

Aug 05 '22 19:08 akemin-dayo

Out of interest, could you give some pointers on what's different about the incorrect BT.601 colours? Maybe I'm just old but I can't tell the difference even when putting them side by side.

Aug 06 '22 20:08 KuleRucket

I… am going to be honest and say that I'm not entirely sure how to answer that question.

Sunshine is performing the RGB→YCbCr colourspace conversion incorrectly (BT.601), while Parsec and NVGFE are performing the RGB→YCbCr colourspace conversion correctly (BT.709).

As an example, let's look at the screenshots I posted above — for demonstration purposes, let's use the winver screenshot.

Open either the Parsec or the NVGFE screenshot above in an image editor, then take a colour picker and get the RGB values of the grey part of the window — it should be #f0f0f0.

Next, take a local screenshot of your own winver window and do the same thing. You should also get the same value of #f0f0f0.

Finally, open the Sunshine screenshot above, and use the colour picker on the same part. You'll get a value of #eef0ed, which is completely incorrect.

Perhaps I may just be more sensitive to colour differences than some people, but to me the issue is immediately noticeable even without having to compare, to the point where Sunshine is pretty much unusable to me because the colour inaccuracy really bothers me ;P

I did spend some time yesterday looking through the codebase a bit (… and staring at a Wireshark packet capture), and it's quite interesting to me that both Sunlight and Moonlight seem to assume BT.601 as the default, when this pretty much (as far as I know) should never be correct when dealing with PC colourspace conversion.

(I've only seen BT.601 be the correct option to use when dealing with SDTV/EDTV analogue video ingest from capture boards, at least in my experience.)

※ Also, I did join the Moonlight Discord server yesterday as well, so feel free to @mention me there in the #sunshine channel if real-time communication would help in getting this resolved at all.

Aug 06 '22 23:08 akemin-dayo

I found some reference to this in the code. 601 is default.

https://github.com/LizardByte/Sunshine/blob/6000b85b1a4ec574d93fbc7545f5bf48f3d5aaa7/src/video.cpp#L883-L913

I am not sure if this is an undocumented config option or not, since I can't find a reference to it in the config.cpp.

Could you try adding encoderCscMode=1 to your config file to see if it uses the 709?

Aug 28 '22 19:08 ReenigneArcher

I am not sure if this is an undocumented config option or not, since I can't find a reference to it in the config.cpp.

It isn't, as far as I'm aware. The config that is being referred to in video.cpp is actually a config_t struct defined here: https://github.com/LizardByte/Sunshine/blob/6000b85b1a4ec574d93fbc7545f5bf48f3d5aaa7/src/video.h#L53-L63 and has no relation to the config file (as you correctly observed).

And just to be sure, I tried what you suggested anyway but there was no change ;P

Aug 28 '22 20:08 akemin-dayo

The colorspace seems to be specified by the client. Testing on my laptop running Linux connecting to a Windows host using software renderering on an AMD GPU, moonlight-qt's logs shows:

00:00:08 - SDL Info (0): FFmpeg-based video decoder chosen

Looking here, it seems that Rec 601 would always be selected if using this renderer.

Sep 02 '22 15:09 psyke83

Sunshine should be doing the Rec 709 -> Rec 601 conversion in the case that the Moonlight client requests it. Maybe that's what's going wrong here.

Whether Rec 709 or Rec 601 is picked shouldn't make a difference in color reproduction for renderers that support both. When streaming from GFE, Rec 709 and Rec 601 generate identical RGB values in Moonlight. All it does is change the coefficients used for the RGB->YUV and YUV->RGB conversions and you end up with the same color in the end.

Sep 03 '22 17:09 cgutman

I can confirm something is not right with colorspace conversion between sunshine/moonlight and the display. Color reproduction is definitely not great.

I put 2 identical screens next to each other, one running a Linux desktop connected natively through displayport, the other connected to a laptop that remotes into it using Moonlight on windows. Comparing side-by-side colors are washed out, which is particularly visible in bright colored text (orange, yellow, green, etc). The problem is not in the Windows laptop because a VNC remote desktop to the same machine has good color reproduction (but terrible performance).

Is there anything anywhere in the chain (host desktop settings - Sunshine - Moonlight - client desktop settings) that could improve this, or does this need to be fixed in either sunshine and/or moonlight?

Oct 07 '22 09:10 w0utert

Scrap my last comment about colorspace conversion, the bad color reproduction I was observing was not actually related to color space handling but to bugs in the RGB to NV12 conversion when using NvFBC with CUDA enabled (see #154)

Oct 10 '22 09:10 w0utert

Please try this build from #723 and see if it addresses the issue.

https://github.com/LizardByte/Sunshine/suites/10239661632/artifacts/502111042

Jan 07 '23 22:01 cgutman

It greatly improves the issue, but there is still a noticeable difference, especially for desktop use. (Basically, the image from Sunshine is still shifted slightly more green than it should be.)

In particular, it still doesn't match the output of NVGFE (or Parsec, though that's less direct of a comparison), although it is much closer now.

I think the most noticeable point of comparison here would be to look at the colour of the white background in the Sunshine web UI / conhost, as well as the colour of that one grey-ish colour Windows loves to use for its dialogs, like in winver.

Moonlight + Sunshine #723 (improved, BT.601)

※ Ground truth (direct screenshots from OS)

※ These screenshots are for reference only — NVGFE, Parsec, and Sunshine will not be able to achieve a perfect match due to the nature of video compression and colourspace conversion from RGB.