[Bisected] Color corruption on intel graphics

[Samsagax]

Being using gamescope on my AMD machine for a while without issues. When trying out on my laptop Nvidia won't let me use it because of #151 but I managed to start gamescope on the intel GPU (iGPU) and then start games from it using the Nvidia GPU (dGPU) by setting some env variables (pretty cool, huh?).

Here is the issue: I started steam client within gamescope on my intel card by forcing VK_ICD_FILENAMES variable like so

$ VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel_icd.x86_64.json gamescope -- steam

The client launches fine but then the colors are corrupted:

But I managed to start a native game (selected Factorio, then Start) and then it all works inside the game (Launched with __NV_PRIME* variables for it to use the dGPU):

Also when the game is launching the window flickers back and forth with the corrupted colors client and the correct colored game. After the game is started, it all "just works".

[l3s2d]

Noticed this after upgrading to 3.10.3 on Arch Linux. Goes away after downgrading to 3.9.5.

[Samsagax]

Noticed this after upgrading to 3.10.3 on Arch Linux. Goes away after downgrading to 3.9.5.

Great. Never noticed this before but with this I'll try to bisect it.

[Samsagax]

Ok I managed to bisect it. Here is the log.

git bisect start
# good: [26aeebf11d2d195fd2fa75f0d85d11dcd3ee5e69] steamcompmgr: publish input change count
git bisect good 26aeebf11d2d195fd2fa75f0d85d11dcd3ee5e69
# bad: [dd22f8db76aa9726b64d7a21095d669aec3b3732] steamcompmgr: Fix validContents nullptr deref
git bisect bad dd22f8db76aa9726b64d7a21095d669aec3b3732
# good: [6402b5cd40c78e0972348133ffe1a57cedbb807c] steamcompmgr: Fix child window cursor barrier
git bisect good 6402b5cd40c78e0972348133ffe1a57cedbb807c
# good: [7e93200e084f49dc8b68b8d0f85ad15afccf2e9e] Dedupe and memoize fb_id and vulkan texture per buffer.
git bisect good 7e93200e084f49dc8b68b8d0f85ad15afccf2e9e
# bad: [296a3d498d02c1277e4de7ed2e67ad537b1d3553] steamcompmgr: Add framework for held commits
git bisect bad 296a3d498d02c1277e4de7ed2e67ad537b1d3553
# bad: [93e4d94a78c1d371f1e2112c6386fe3dfd085bb9] steamcompmgr: Add --cursor-hotspot
git bisect bad 93e4d94a78c1d371f1e2112c6386fe3dfd085bb9
# bad: [4ee56a1e75a043c6f317ea2f71c96b2061e85cf5] rendervulkan: Do not premultiply color
git bisect bad 4ee56a1e75a043c6f317ea2f71c96b2061e85cf5
# bad: [23430084e2245e226f27dbe1882bd30633a1d50c] rendervulkan: Blend in linear space
git bisect bad 23430084e2245e226f27dbe1882bd30633a1d50c
# first bad commit: [23430084e2245e226f27dbe1882bd30633a1d50c] rendervulkan: Blend in linear space

And the first bad commit:

23430084e2245e226f27dbe1882bd30633a1d50c is the first bad commit
commit 23430084e2245e226f27dbe1882bd30633a1d50c
Author: Joshua Ashton <[email protected]>
Date:   Sat Nov 13 22:28:25 2021 +0000

    rendervulkan: Blend in linear space

    We must blend in linear space, not SRGB space or we get these horrible browns.

    Gets us closer to how this looks with planes.

 src/composite.comp   | 28 ++++++++++++++++++++++++++--
 src/rendervulkan.cpp | 27 ++++++++++++++-------------
 2 files changed, 40 insertions(+), 15 deletions(-)

[misyltoad]

Once we fix validation stuff and add acquire barrier itd make sense to open an issue on Mesa if it isnt fixed.

[leonidx86]

Reporting my test on two different Intel iGPUs, maybe this will help to locate the issue. For both tests the Linux distro and package versions are the same:

• Kernel: 5.16.3.arch1-1
• Mesa: 21.3.5-1
• Gamescope: 3.11.0beta4-1

gamescope -W 320 -H 320 -m 1 glxgears

GPU: Intel(R) HD Graphics 4000 (IVB GT2)

GPU: Intel(R) UHD Graphics 630 (CFL GT2)

[Samsagax]

Might worth a try with mesa-git package to see if it's fixed upstream.

[leonidx86]

Might worth a try with mesa-git package to see if it's fixed upstream.

Tested now, same result with mesa-git (version 22.0.0_devel.149533.24fef8f33da.d41d8cd98f00b204e9800998ecf8427e-1).

[DadSchoorse]

This should be reported to mesa, as far as I know we do everything correctly now. All the validation errors have been fixed.

[vignedev]

Not sure how helpful this might be, but I've been experimenting/hacking a few changes to try to get it visually working on my setup.

There are two scenarios that I have tested, both involving changes in the s_DRMVKFormatTable structure and both resulting in a non-glitch state, however with different bugs:

---

1. Forcing both vkFormat and vkFormatSrgb to be their sRGB variant

struct {
	uint32_t DRMFormat;
	VkFormat vkFormat;
	VkFormat vkFormatSrgb;
	bool bHasAlpha;
} s_DRMVKFormatTable[] = {
-       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, true },
-       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, false },
+       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_SRGB, VK_FORMAT_B8G8R8A8_SRGB, true },
+       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_SRGB, VK_FORMAT_B8G8R8A8_SRGB, false },
	{ DRM_FORMAT_NV12, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, false },
	{ DRM_FORMAT_INVALID, VK_FORMAT_UNDEFINED, VK_FORMAT_UNDEFINED, false },
};

This makes gamescope run with correct colors, however enabling FSR results in black screen.

Without FSR	With FSR

---

2. Forcing both vkFormat and vkFormatSrgb to be their UNNORM variant

struct {
	uint32_t DRMFormat;
	VkFormat vkFormat;
	VkFormat vkFormatSrgb;
	bool bHasAlpha;
} s_DRMVKFormatTable[] = {
-       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, true },
-       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_SRGB, false },
+       { DRM_FORMAT_ARGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_UNORM, true },
+       { DRM_FORMAT_XRGB8888, VK_FORMAT_B8G8R8A8_UNORM, VK_FORMAT_B8G8R8A8_UNORM, false },

	{ DRM_FORMAT_NV12, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, false },
	{ DRM_FORMAT_INVALID, VK_FORMAT_UNDEFINED, VK_FORMAT_UNDEFINED, false },
};

This makes gamescope run in incorrect washed-out colors (obviously due to color conversion), however when using FSR all colors return into their properness. This however required the revertion of changes brought by eff99fdf813c199e360d73b0b5a626f048f7f259 to get it working.

Without FSR	With FSR

I'm not experienced at all in these regions and I found these changes purely by trial and error, so I am unsure of any possible side-effects nor the full purpose of such changes, however I hope it will help narrow the issue down somewhat.

[patrickaldis]

I can confirm I'm having near identical same issues on an 7700HQ under intel UHD 630

Created an issue here, but realising this is actually the same as this thread

https://github.com/Plagman/gamescope/issues/393

[DadSchoorse]

@vignedev All of that indicates that it's a driver issue, so something that should be reported to mesa.

[patrickaldis]

@DadSchoorse Would this really be a mesa issue if the game runs without gamescope on mesa?

[DadSchoorse]

@patrickaldis There's always the possibility that there's some subtle bug in our code, but given that gamescope works fine on radv I think it's an anv problem. The hacks by @vignedev show that there's is some issue with mutable images + dmabufs and with the import barriers. The game doesn't matter.

[leonidx86]

As an anecdotal evidence, I've tested this on several different Intel CPUs with integrated graphics. Only one of them (UHD 630) has this visual artifact (but works fine without Gamescope). Everything else works fine with or without Gamescope.

@Samsagax Which Intel CPU/iGPU have you tested on? @l3s2d Says the issue is not present on v3.9.5, but I can't downgrade, since there is no package below v3.11.0 at https://archive.archlinux.org/packages/g/gamescope/.

[ccajas]

I'd like to add my two cents in, and say that these problems occur on most games I've tried, but there are at least a few games where Gamescope is mostly working fine with Intel UHD graphics. I'm running it on a Celeron N4120 with UHD 600 graphics.

I've tested out Portal and Lara Croft: Temple of Osiris and both these games are playable with Gamescope, at least in all the sections with real-time 3D graphics.

Portal is very playable, only seeing image artifacts in the Valve splash screen and the death sequences. Temple of Osiris starts with a setup window where artifacts show up, but once you get past that window it all looks fine.

[Samsagax]

As an anecdotal evidence, I've tested this on several different Intel CPUs with integrated graphics. Only one of them (UHD 630) has this visual artifact (but works fine without Gamescope). Everything else works fine with or without Gamescope.

@Samsagax Which Intel CPU/iGPU have you tested on? @l3s2d Says the issue is not present on v3.9.5, but I can't downgrade, since there is no package below v3.11.0 at https://archive.archlinux.org/packages/g/gamescope/.

I have an i5 6200U cpu, the iGPU is a 520 (SKL GT2).

[Samsagax]

For what is worth, seems to be a problem with the vulkan bit of the intel driver for those iGPUs. Games that use OpenGL will not have this issue.

Does anyone have an account on the mesa gitlab instance to report this? I think we colected valuable information and should be able to report it to be fixed easily.

@l3s2d Says the issue is not present on v3.9.5, but I can't downgrade, since there is no package below v3.11.0 at https://archive.archlinux.org/packages/g/gamescope/.

You would need to use git tags for this. Use the gamescope-git package as a reference.

[Oschowa]

I can reproduce this issue on my Kaby Lake UHD 620 using Mesa 21.3.5. However, on gamescope master and latest Mesa main it's working fine. INTEL_DEBUG=norbc also makes it work fine on Mesa 21.3.5 (it will however likely reduce performance).
This Mesa commit that fixes the issue is https://gitlab.freedesktop.org/mesa/mesa/-/commit/ca791f5c5d88a6451e4934f3e03fa0e38a7759fa

[leonidx86]

@Oschowa You're right! With INTEL_DEBUG=norbc it works fine. However, with latest Mesa (22.1.0) and relatively latest Gamescope (3.11.0beta6-1) the issue is still there on UHD 630.

[Oschowa]

Yes, it seems like latest Mesa only fixes it for vulkan apps running in gamescope, anything using OpenGL is still broken. I opened a Mesa issue about this here https://gitlab.freedesktop.org/mesa/mesa/-/issues/6029

[JordanViknar]

I can confirm I also encounter this issue running Minecraft Java Edition or glxgears on my Intel HD Graphics 500 iGPU.

[patrickaldis]

The artifacts in my case seem to be more extreme:

[adolfintel]

I also have this problem on Intel UHD 600 (Gemini Lake)

[Samsagax]

I'm currently testing some stuff and the issue seems to be fixed. Could anyone please confirm? Relevant package versions:

• linux-zen 5.16.16.zen1-1
• mesa 21.3.7-2
• vulkan-intel 21.3.7-2
• vulkan-icd-loader 1.3.208-1

[leonidx86]

@Samsagax Nope. Even installed the Zen Kernel. The only major difference with your config seems to be the GPU.

• GPU Intel(R) UHD Graphics 630 (CFL GT2)
• linux 5.16.16-arch1-1 and 5.16.16-zen1-1-zen
• mesa 21.3.7-2
• vulkan-intel 21.3.7-2
• vulkan-icd-loader 1.3.208-1

[JordanViknar]

I'm currently testing some stuff and the issue seems to be fixed. Could anyone please confirm? Relevant package versions:

• linux-zen 5.16.16.zen1-1
• mesa 21.3.7-2
• vulkan-intel 21.3.7-2
• vulkan-icd-loader 1.3.208-1

The issue still occurs for me, using the latest Arch Linux packages.

[JordanViknar]

Just noticed, but the issue also happens on my other laptop (which doesn't really need it, unlike my main one) still with the latest packages, but with this Intel card :

• Intel CoffeeLake-H GT2 [UHD Graphics 630]

That laptop ALSO has an NVIDIA card, and I used the same workaround. Doesn't seem to be related though.

[shmugoh]

Intel HD Graphics 620 iGPU is also affected by this issue. A bit unrelated but might aswell say it. This pops up at the start

[Samsagax]

The CAP_SYS_NICE is unrelated for sure. But the other option might not be. I'll test it and report back.

[leonidx86]

The sysctl setting is also unrelated as far as I know. echo "dev.i915.perf_stream_paranoid = 0" > /etc/sysctl.d/60-mdapi.conf and reboot. More info: https://www.intel.com/content/www/us/en/develop/documentation/vtune-help/top/installation/set-up-system-for-gpu-analysis.html