Gamescope coredumps upon trying to start it in Hyprland

[Andy3153]

Current setup is Arch with linux-zen, nvidia-dkms, hyprland-git, gamescope on a hybrid GPU laptop, if any of it matters.

Upon trying to run Gamescope, this error gets thrown:

No CAP_SYS_NICE, falling back to regular-priority compute and threads.
Performance will be affected.
wlserver: [backend/headless/backend.c:68] Creating headless backend
vulkan: physical device 10de:2520 compute queue doesn't support presenting on our surface, using graphics queue
vulkan: selecting physical device 'NVIDIA GeForce RTX 3060 Laptop GPU': queue family 0 (general queue family 0)
vulkan: physical device supports DRM format modifiers
zsh: segmentation fault (core dumped)  gamescope

The result is identical to running gamescope with the --expose-wayland option.

[sharkautarch]

Can confirm the bug with nvidia I tried reverting commit https://github.com/ValveSoftware/gamescope/commit/bd722f786f5a74f8282d903bd5eb3d331c4cd920 which was thought to be the commit where this issue started, as discusssed in https://github.com/ValveSoftware/gamescope/issues/948. It actually seems like reverting just that commit (after fixing the compiler errors reverting it caused) makes gamescope work on nvidia again

Though I do have to run gamescope with --prefer-vk-device in order for it to run You can find the ID of your gpu by running: MESA_VK_DEVICE_SELECT=list vulkaninfo (vulkaninfo is in the vulkan-tools package for archlinux, debian and ubuntu)

I am on a hybrid graphics nvidia/intel laptop, but running nvidia-smi also confirms that gamescope is running on the nvidia gpu for me. On the other hand, now I have the opposite problem: when trying to run gamescope on anything but my nvidia gpu, it crashes!

Here's a fork I made with the revert: https://github.com/sharkautarch/gamescope/tree/nvidia-fix note that the revert is only in the 'nvidia-fix' branch of it Press the 'Download Zip' button to download from the 'nvidia-fix' branch, or clone the fork and then do cd gamescope; git checkout nvidia-fix

[fikri-izzuddin]

thanks @sharkautarch 🙏 that did fix the problem

[weidtn]

Any possibility that this commit bd722f7 gets reverted in this repo? Somehow I get terrible fps when using @sharkautarch 's fix.

[sharkautarch]

Any possibility that this commit bd722f7 gets reverted in this repo? Somehow I get terrible fps when using @sharkautarch 's fix.

@weidtn From what I understand, commit https://github.com/ValveSoftware/gamescope/commit/bd722f786f5a74f8282d903bd5eb3d331c4cd920 triggers the nvidia driver bug (for reference, see: https://github.com/ValveSoftware/gamescope/issues/948)

If you're having bad fps/framepacing, try building from my nvidia-fix-and-vblank-debug-extra-experimental-v4.7 branch also you could try running gamescope like so: MESA_VK_WSI_PRESENT_MODE=mailbox gamescope -f --prefer-vk-device <id> ... -- <game>

[weidtn]

I tried this running vkcube and glxgears. It is a stuttery mess. Why is this commit not reverted in the main branch here? Is it that important?

[slmb3]

I get segfault and this backtrace in the main branch just by starting gamescope

#0  0x00007fdf19dccde0 in  () at /usr/lib/libnvidia-glcore.so.545.29.06
#1  0x00007fdf19d1e194 in  () at /usr/lib/libnvidia-glcore.so.545.29.06
#2  0x00007fdf20770db0 in  () at /usr/lib/libGLX_nvidia.so.0
#3  0x00007fdf2744f2a0 in terminator_CreateDevice
    (physicalDevice=<optimized out>, pCreateInfo=<optimized out>, pAllocator=<optimized out>, pDevice=<optimized out>)
    at /usr/src/debug/vulkan-icd-loader/Vulkan-Loader-1.3.269/loader/loader.c:5841
#4  0x00007fdf27445bb4 in loader_create_device_chain
    (pd=pd@entry=0x563b8d324220, pCreateInfo=pCreateInfo@entry=0x7fff8251dad0, pAllocator=pAllocator@entry=0x0, inst=inst@entry=0x563b8d418b90, dev=dev@entry=0x563b8d607600, callingLayer=callingLayer@entry=0x0, layerNextGDPA=<optimized out>)
    at /usr/src/debug/vulkan-icd-loader/Vulkan-Loader-1.3.269/loader/loader.c:4949
#5  0x00007fdf274475c5 in loader_layer_create_device
    (instance=<optimized out>, physicalDevice=<optimized out>, pCreateInfo=<optimized out>, pAllocator=<optimized out>, pDevice=<optimized out>, layerGIPA=<optimized out>, nextGDPA=<optimized out>)
    at /usr/src/debug/vulkan-icd-loader/Vulkan-Loader-1.3.269/loader/loader.c:4323
#6  0x00007fdf2745c61b in vkCreateDevice
    (physicalDevice=0x563b8d3039e0, pCreateInfo=0x7fff8251dad0, pAllocator=0x0, pDevice=0x563b8cbe1570 <_ZL8g_device.lto_priv.0+656>)
    at /usr/src/debug/vulkan-icd-loader/Vulkan-Loader-1.3.269/loader/trampoline.c:953
#7  0x0000563b8cacb6ec in CVulkanDevice::createDevice() (this=<optimized out>) at ../gamescope/src/rendervulkan.cpp:582
#8  0x0000563b8cacc3f9 in CVulkanDevice::BInit(VkInstance_T*, VkSurfaceKHR_T*) [clone .constprop.0]
    (instance=0x563b8d418b90, surface=0x563b8d38f6e0, this=<optimized out>) at ../gamescope/src/rendervulkan.cpp:251
#9  0x0000563b8ca47e73 in vulkan_init(VkInstance_T*, VkSurfaceKHR_T*) (instance=0x0, surface=0x2)
    at ../gamescope/src/rendervulkan.cpp:3228
#10 0x0000563b8c9e48a7 in main(int, char**) (argc=<optimized out>, argv=<optimized out>) at ../gamescope/src/main.cpp:786

Using your nvidia-fix branches, it fixes the segfault, but after running i.e. glxgears it's laggy mess after a few seconds. sometimes it's good for 10 seconds or longer and then become laggy mess.

no combination of nvidia-fix-and-vblank-debug-extra-experimental-v4.7 (nor 4.8) branch, MESA_VK_WSI_PRESENT_MODE=mailbox, ENABLE_GAMESCOPE_WSI=0 seemed to help

Let me know if any more info would help or other debugging steps to try

I also see there's similar discussion on https://github.com/ValveSoftware/gamescope/issues/948

[weidtn]

Not sure what fixed it, but the stutter and segfault are gone now