gamescope icon indicating copy to clipboard operation
gamescope copied to clipboard

gamescope crash on ubuntu 23.10

Open djdeath opened this issue 3 months ago • 10 comments

Is there a particular version requirement on libdecor maybe?

Thread 8 "gamescope-xwm" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe2ffd6c0 (LWP 502187)]
wl_proxy_get_version (proxy=0x0) at ../src/wayland-client.c:2289
2289		return proxy->version;
(gdb) bt
#0  wl_proxy_get_version (proxy=0x0) at ../src/wayland-client.c:2289
#1  0x00007ffff79328ce in ??? () at /lib/x86_64-linux-gnu/libdecor-0.so.0
#2  0x00007ffff793367a in libdecor_frame_set_visibility () at /lib/x86_64-linux-gnu/libdecor-0.so.0
#3  0x000055555566bbeb in gamescope::CWaylandPlane::Init (this=0x555555a7e258, pParent=0x0, pSiblingBelow=0x0) at ../src/wayland_backend.cpp:570
#4  0x000055555566d140 in gamescope::CWaylandBackend::PostInit (this=0x555555a7e1f0) at ../src/wayland_backend.cpp:893
#5  0x00005555555bf32a in steamcompmgr_main (argc=2, argv=0x7fffffffd5d8) at ../src/steamcompmgr.cpp:7568
#6  0x00005555555f68de in steamCompMgrThreadRun (argc=2, argv=0x7fffffffd5d8) at ../src/main.cpp:921
#7  0x00005555555f73b9 in std::__invoke_impl<void, void (*)(int, char**), int, char**> (__f=@0x555555c1f138: 0x5555555f68a0 <steamCompMgrThreadRun(int, char**)>)
    at /usr/include/c++/13/bits/invoke.h:61
#8  0x00005555555f72e2 in std::__invoke<void (*)(int, char**), int, char**> (__fn=@0x555555c1f138: 0x5555555f68a0 <steamCompMgrThreadRun(int, char**)>)
    at /usr/include/c++/13/bits/invoke.h:96
#9  0x00005555555f7201 in std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::_M_invoke<0ul, 1ul, 2ul> (this=0x555555c1f128) at /usr/include/c++/13/bits/std_thread.h:292
#10 0x00005555555f719a in std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::operator() (this=0x555555c1f128) at /usr/include/c++/13/bits/std_thread.h:299
#11 0x00005555555f717a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> > >::_M_run (this=0x555555c1f120)
    at /usr/include/c++/13/bits/std_thread.h:244
#12 0x00007ffff76e6333 in ??? () at /lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x00007ffff7297b5a in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
#14 0x00007ffff73285fc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

djdeath avatar Mar 10 '24 22:03 djdeath

There shouldn't be, but I am also not seeing libdecor_frame_set_visibility work properly anyway, but it also isn't working in their demo app...

            // !!! Health Warning: This below call doesn't work on Plasma for borderless, and just makes the window not show up....
            // Why? I don't know. Doesn't work in the demo app either. /shrug

Joshua-Ashton avatar Mar 10 '24 22:03 Joshua-Ashton

Try latest master? I think I worked around it?

Borderless still doesn't work for me and libdecor_frame_set_visibility does nothing :/

Joshua-Ashton avatar Mar 10 '24 22:03 Joshua-Ashton

Thanks, slightly different backtrace, but still segfaulting :

Thread 8 "gamescope-xwm" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe2ffd6c0 (LWP 506941)]
wl_proxy_get_version (proxy=0x0) at ../src/wayland-client.c:2289
2289		return proxy->version;
(gdb) bt
#0  wl_proxy_get_version (proxy=0x0) at ../src/wayland-client.c:2289
#1  0x00007ffff7933c89 in libdecor_frame_commit () at /lib/x86_64-linux-gnu/libdecor-0.so.0
#2  0x000055555566c0a2 in gamescope::CWaylandPlane::CommitLibDecor
    (this=0x555555a7e258, pConfiguration=0x0) at ../src/wayland_backend.cpp:661
#3  0x000055555566bba9 in gamescope::CWaylandPlane::Init
    (this=0x555555a7e258, pParent=0x0, pSiblingBelow=0x0) at ../src/wayland_backend.cpp:568
#4  0x000055555566d152 in gamescope::CWaylandBackend::PostInit (this=0x555555a7e1f0)
    at ../src/wayland_backend.cpp:897
#5  0x00005555555bf32a in steamcompmgr_main (argc=2, argv=0x7fffffffd5c8)
    at ../src/steamcompmgr.cpp:7568
#6  0x00005555555f68de in steamCompMgrThreadRun (argc=2, argv=0x7fffffffd5c8)
    at ../src/main.cpp:921
#7  0x00005555555f73b9 in std::__invoke_impl<void, void (*)(int, char**), int, char**>
    (__f=@0x555555bf2bf8: 0x5555555f68a0 <steamCompMgrThreadRun(int, char**)>)
    at /usr/include/c++/13/bits/invoke.h:61
#8  0x00005555555f72e2 in std::__invoke<void (*)(int, char**), int, char**>
    (__fn=@0x555555bf2bf8: 0x5555555f68a0 <steamCompMgrThreadRun(int, char**)>)
    at /usr/include/c++/13/bits/invoke.h:96
#9  0x00005555555f7201 in std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::_M_invoke<0ul, 1ul, 2ul> (this=0x555555bf2be8)
    at /usr/include/c++/13/bits/std_thread.h:292
#10 0x00005555555f719a in std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::operator() (this=0x555555bf2be8) at /usr/include/c++/13/bits/std_thread.h:299
#11 0x00005555555f717a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> > >::_M_run (this=0x555555bf2be0)
    at /usr/include/c++/13/bits/std_thread.h:244
#12 0x00007ffff76e6333 in ??? () at /lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x00007ffff7297b5a in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
#14 0x00007ffff73285fc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

djdeath avatar Mar 10 '24 23:03 djdeath

@djdeath

#0  wl_proxy_get_version (proxy=0x0) at ../src/wayland-client.c:2289
#1  0x00007ffff7933c89 in libdecor_frame_commit () at /lib/x86_64-linux-gnu/libdecor-0.so.0

hmm I wonder if this issue is relevant: https://gitlab.freedesktop.org/libdecor/libdecor/-/issues/64

also maybe you should try running w/ valgrind to see if that provides more information: valgrind -v --tool=memcheck --keep-debuginfo=yes --read-inline-info=yes --read-var-info=yes --sim-hints=lax-ioctls --fair-sched=yes --leak-check=no --track-origins=yes --num-transtab-sectors=32 -- gamescope <insert remaining stuff here>

You might need to install these packages, if you don't have them already, in order to run valgrind:

  • valgrind
  • debuginfod

sharkautarch avatar Mar 11 '24 15:03 sharkautarch

==75299== Thread 10 gamescope-xwm:
==75299== Invalid read of size 4
==75299==    at 0x4860AB6: wl_proxy_get_version (wayland-client.c:2289)
==75299==    by 0x4EE18CD: UnknownInlinedFun (xdg-shell-client-protocol.h:1175)
==75299==    by 0x4EE18CD: frame_set_window_geometry (libdecor.c:278)
==75299==    by 0x4EE2679: libdecor_frame_set_visibility (libdecor.c:629)
==75299==    by 0x21FBEA: gamescope::CWaylandPlane::Init(gamescope::CWaylandPlane*, gamescope::CWaylandPlane*) (wayland_backend.cpp:570)
==75299==    by 0x22113F: gamescope::CWaylandBackend::PostInit() (wayland_backend.cpp:893)
==75299==    by 0x173329: steamcompmgr_main(int, char**) (steamcompmgr.cpp:7568)
==75299==    by 0x1AA8DD: steamCompMgrThreadRun(int, char**) (main.cpp:921)
==75299==    by 0x1AB3B8: void std::__invoke_impl<void, void (*)(int, char**), int, char**>(std::__invoke_other, void (*&&)(int, char**), int&&, char**&&) (invoke.h:61)
==75299==    by 0x1AB2E1: std::__invoke_result<void (*)(int, char**), int, char**>::type std::__invoke<void (*)(int, char**), int, char**>(void (*&&)(int, char**), int&&, char**&&) (invoke.h:96)
==75299==    by 0x1AB200: void std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) (std_thread.h:292)
==75299==    by 0x1AB199: std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::operator()() (std_thread.h:299)
==75299==    by 0x1AB179: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> > >::_M_run() (std_thread.h:244)
==75299==  Address 0x40 is not stack'd, malloc'd or (recently) free'd
==75299== 
==75299== 
==75299== Process terminating with default action of signal 11 (SIGSEGV)
==75299==  Access not within mapped region at address 0x40
==75299==    at 0x4860AB6: wl_proxy_get_version (wayland-client.c:2289)
==75299==    by 0x4EE18CD: UnknownInlinedFun (xdg-shell-client-protocol.h:1175)
==75299==    by 0x4EE18CD: frame_set_window_geometry (libdecor.c:278)
==75299==    by 0x4EE2679: libdecor_frame_set_visibility (libdecor.c:629)
==75299==    by 0x21FBEA: gamescope::CWaylandPlane::Init(gamescope::CWaylandPlane*, gamescope::CWaylandPlane*) (wayland_backend.cpp:570)
==75299==    by 0x22113F: gamescope::CWaylandBackend::PostInit() (wayland_backend.cpp:893)
==75299==    by 0x173329: steamcompmgr_main(int, char**) (steamcompmgr.cpp:7568)
==75299==    by 0x1AA8DD: steamCompMgrThreadRun(int, char**) (main.cpp:921)
==75299==    by 0x1AB3B8: void std::__invoke_impl<void, void (*)(int, char**), int, char**>(std::__invoke_other, void (*&&)(int, char**), int&&, char**&&) (invoke.h:61)
==75299==    by 0x1AB2E1: std::__invoke_result<void (*)(int, char**), int, char**>::type std::__invoke<void (*)(int, char**), int, char**>(void (*&&)(int, char**), int&&, char**&&) (invoke.h:96)
==75299==    by 0x1AB200: void std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) (std_thread.h:292)
==75299==    by 0x1AB199: std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> >::operator()() (std_thread.h:299)
==75299==    by 0x1AB179: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(int, char**), int, char**> > >::_M_run() (std_thread.h:244)

djdeath avatar Mar 11 '24 16:03 djdeath

==75299== Thread 10 gamescope-xwm:
==75299== Invalid read of size 4
==75299==    at 0x4860AB6: wl_proxy_get_version (wayland-client.c:2289)
==75299==    by 0x4EE18CD: UnknownInlinedFun (xdg-shell-client-protocol.h:1175)
==75299==    by 0x4EE18CD: frame_set_window_geometry (libdecor.c:278)
==75299==    by 0x4EE2679: libdecor_frame_set_visibility (libdecor.c:629)

hmmm, for libdecor.c function frame_set_window_geometry, valgrind gives line 278 Since @djdeath is on ubuntu 23.10, looking at https://packages.ubuntu.com/mantic/allpackages indicates they are using libdecor version 0.1.1-2 And looking here: https://gitlab.freedesktop.org/libdecor/libdecor/-/blob/0.1.1/src/libdecor.c?ref_type=tags#L278 seems to indicate that libdecor is crashing upon calling xdg_surface_set_window_geometry() ... and searching for a header (it is generated from the xdg shell protocol xml) matching xdg-shell-client-protocol.h, I get:

static inline void
xdg_surface_set_window_geometry(struct xdg_surface *xdg_surface, int32_t x, int32_t y, int32_t width, int32_t height)
{
        wl_proxy_marshal_flags((struct wl_proxy *) xdg_surface,
                         XDG_SURFACE_SET_WINDOW_GEOMETRY, NULL, wl_proxy_get_version((struct wl_proxy *) xdg_surface), 0, x, y, width, height);
}

the above generated xdg_surface_set_window_geometry() function might not be the same on ubuntu 23.10 tho

So seems like there's an issue where the libdecor+wayland/xdg protocol code tries to cast struct xdg_surface* to struct wl_proxy *, and then tries to access the version member of struct wl_proxy, and somehow that causes a segfault... Looks like version member of wl_proxy is only initialized when:

  • calling some sort of init function for libdecor
  • indirectly: for wayland, when a wl_proxy is made for a display after display object was made w/ function wl_display_connect_to_fd (version automatically set to 0)

Idk if the issue is that the a version member for xdg_surface is never initialized, or if it was initialized/attempted to be initialized, but somehow maybe a memory/pointer alignment issue or race condition (or maybe even UB around pointer aliasing within structs??? idk) is causing it to result in a segfault

sharkautarch avatar Mar 11 '24 18:03 sharkautarch

@Joshua-Ashton Looking at how GLFW handles libdecor, it seems like they call some sort of libdecor new/init function, when they first initialize their wayland backend: https://github.com/glfw/glfw/blob/dc557ecf38a42b0b93898a7aef69f6dc48bf0e57/src/wl_init.c#L842 Maybe gamescope needs to just do a similar thing on init?

sharkautarch avatar Mar 12 '24 15:03 sharkautarch

Is there any workaround for this issue?

djdeath avatar Mar 27 '24 05:03 djdeath

Is there any workaround for this issue?

I could try making a package build script for building gamescope against a statically linked newer version of libdecor I guess

not sure if that would actually fix the problem for you tho

sharkautarch avatar Mar 27 '24 12:03 sharkautarch

@djdeath Now that I think about it, you could just try building gamescope from this tag I have: https://github.com/sharkautarch/gamescope/commits/3.14.21

Or if you're okay with a modified version of gamescope, you could try out my test_w_no_lut branch which simply has a collection of tweaks to improve performance on igpus:

  • see https://github.com/ValveSoftware/gamescope/issues/1221#issuecomment-2042873559 for build/install instructions

sharkautarch avatar Apr 08 '24 14:04 sharkautarch