xrdp icon indicating copy to clipboard operation
xrdp copied to clipboard

Published 20 hours ago • verified publisher icon neutrinolabs

Ubuntu Crash Messages for /usr/bin/xrdp

Open noorez opened this issue 2 years ago • 13 comments

When logging into an Ubuntu 22.04 session via Remote Desktop, I occasionally receive the Ubuntu Apport Crash Dialog telling me /usr/bin/xrdp has crashed, although there doesn't appear to be any functionality problems. Here is the log output from one login:

[20220920-21:59:05] [INFO ] Socket 12: connection accepted from [::ffff:192.168.0.10]:62586
[20220920-21:59:05] [INFO ] Security protocol: configured [SSL], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [SSL]
[20220920-21:59:05] [ERROR] SSL_read: I/O error
[20220920-21:59:05] [ERROR] libxrdp_force_read: header read error
[20220920-21:59:05] [ERROR] Processing [ITU-T T.125] Connect-Initial failed
[20220920-21:59:05] [ERROR] [MCS Connection Sequence] receive connection request failed
[20220920-21:59:05] [ERROR] xrdp_sec_incoming: xrdp_mcs_incoming failed
[20220920-21:59:05] [ERROR] xrdp_rdp_incoming: xrdp_sec_incoming failed
[20220920-21:59:05] [ERROR] xrdp_process_main_loop: libxrdp_process_incoming failed
[20220920-21:59:05] [ERROR] xrdp_iso_send: trans_write_copy_s failed
[20220920-21:59:05] [ERROR] Sending [ITU T.125] DisconnectProviderUltimatum failed
[20220920-21:59:05] [INFO ] Socket 12: connection accepted from [::ffff:192.168.0.10]:62587
[20220920-21:59:05] [INFO ] Security protocol: configured [SSL], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [SSL]
[20220920-21:59:05] [INFO ] Connected client computer name:  noorez
[20220920-21:59:05] [WARN ] Physical desktop dimensions (0x0) are invalid
[20220920-21:59:05] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc006 is unknown (ignored)
[20220920-21:59:05] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc00a is unknown (ignored)
[20220920-21:59:05] [INFO ] xrdp_load_keyboard_layout: Keyboard information sent by the RDP client, keyboard_type:[0x04], keyboard_subtype:[0x00], keylayout:[0x00000409]
[20220920-21:59:05] [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [us] options []
[20220920-21:59:05] [INFO ] TLS connection established from [::ffff:192.168.0.10]:62587 TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384
[20220920-21:59:05] [INFO ] xrdp_caps_process_pointer: client supports new(color) cursor
[20220920-21:59:05] [INFO ] xrdp_process_offscreen_bmpcache: support level 1 cache size 10485760 MB cache entries 100
[20220920-21:59:05] [INFO ] xrdp_caps_process_codecs: nscodec, codec id 1, properties len 3
[20220920-21:59:05] [WARN ] xrdp_caps_process_codecs: unknown codec id 5
[20220920-21:59:05] [INFO ] xrdp_caps_process_codecs: RemoteFX, codec id 3, properties len 49
[20220920-21:59:05] [INFO ] Loading keymap file /etc/xrdp/km-00000409.ini
[20220920-21:59:05] [WARN ] local keymap file for 0x00000409 found and doesn't match built in keymap, using local keymap file
[20220920-21:59:05] [INFO ] connecting to sesman on sesman.socket
[20220920-21:59:05] [WARN ] Ignoring obsolete SCP port value '3350'
[20220920-21:59:05] [INFO ] xrdp_wm_log_msg: sesman connect ok
[20220920-21:59:05] [INFO ] sesman connect ok
[20220920-21:59:05] [INFO ] sending login info to session manager. Please wait...
[20220920-21:59:05] [INFO ] xrdp_wm_log_msg: login successful for user noorez on display 10
[20220920-21:59:05] [INFO ] login successful for user noorez on display 10
[20220920-21:59:05] [INFO ] loaded module 'libxup.so' ok, interface size 10408, version 4
[20220920-21:59:05] [INFO ] started connecting
[20220920-21:59:05] [INFO ] lib_mod_connect: connecting via UNIX socket
[20220920-21:59:07] [INFO ] lib_mod_log_peer: xrdp_pid=6968 connected to X11rdp_pid=7035 X11rdp_uid=1000 X11rdp_gid=1000 client=[::ffff:192.168.0.10]:62587
[20220920-21:59:07] [INFO ] connected ok

noorez avatar Sep 21 '22 04:09 noorez

Log file looks fine.

Have you got anything in /var/crash?

matt335672 avatar Sep 23 '22 09:09 matt335672

_usr_sbin_xrdp.130.txt

attached is the crash file

noorez avatar Sep 23 '22 16:09 noorez

Notes:

  • Hyper-V with vsock:// connection (so Remote Desktop via the hyper-v interface)

It looks like it crashes with SIGSEGV ? .. I'm not sure if this is related, but when logging out of a session, I'm not sure if the disconnect is happening gracefully?.. Hyper-V gives the warning message: "The session was disconnected. If you want to continue, try to connect again...."

noorez avatar Sep 23 '22 16:09 noorez

copy from log file /var/log/xrdp.log

[20220923-11:34:37] [INFO ] address [-1] port [3389] mode 3
[20220923-11:34:37] [INFO ] listening to port 3389 on -1
[20220923-11:34:37] [INFO ] xrdp_listen_pp done
[20220923-11:34:39] [INFO ] starting xrdp with pid 3662
[20220923-11:34:39] [INFO ] address [-1] port [3389] mode 3
[20220923-11:34:39] [INFO ] listening to port 3389 on -1
[20220923-11:34:39] [INFO ] xrdp_listen_pp done
[20220923-11:34:44] [INFO ] Socket 12: connection accepted from AF_VSOCK:cid=2/port=2261028320
[20220923-11:34:44] [INFO ] Security protocol: configured [RDP], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [RDP]
[20220923-11:34:44] [ERROR] libxrdp_force_read: header read error
[20220923-11:34:44] [ERROR] Processing [ITU-T T.125] Connect-Initial failed
[20220923-11:34:44] [INFO ] Socket 12: connection accepted from AF_VSOCK:cid=2/port=2261028321
[20220923-11:34:44] [ERROR] [MCS Connection Sequence] receive connection request failed
[20220923-11:34:44] [INFO ] Security protocol: configured [RDP], requested [RDP], selected [RDP]
[20220923-11:34:44] [ERROR] xrdp_sec_incoming: xrdp_mcs_incoming failed
[20220923-11:34:44] [INFO ] Connected client computer name: EDNOOREZK
[20220923-11:34:44] [ERROR] xrdp_rdp_incoming: xrdp_sec_incoming failed
[20220923-11:34:44] [WARN ] Client and server do not both support the same encryption.
[20220923-11:34:44] [ERROR] xrdp_process_main_loop: libxrdp_process_incoming failed
[20220923-11:34:44] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc006 is unknown (ignored)
[20220923-11:34:44] [ERROR] xrdp_iso_send: trans_write_copy_s failed
[20220923-11:34:44] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc00a is unknown (ignored)
[20220923-11:34:44] [ERROR] Sending [ITU T.125] DisconnectProviderUltimatum failed
[20220923-11:34:44] [INFO ] xrdp_load_keyboard_layout: Keyboard information sent by the RDP client, keyboard_type:[0x04], keyboard_subtype:[0x00], keylayout:[0x00000409]
[20220923-11:34:44] [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [us] options []
[20220923-11:34:44] [INFO ] Non-TLS connection established from AF_VSOCK:cid=2/port=2261028321 with security level : none
[20220923-11:34:44] [INFO ] xrdp_caps_process_pointer: client supports new(color) cursor
[20220923-11:34:44] [INFO ] xrdp_process_offscreen_bmpcache: support level 1 cache size 10485760 MB cache entries 100
[20220923-11:34:44] [INFO ] xrdp_caps_process_codecs: nscodec, codec id 1, properties len 3
[20220923-11:34:44] [WARN ] xrdp_caps_process_codecs: unknown codec id 5
[20220923-11:34:44] [INFO ] xrdp_caps_process_codecs: RemoteFX, codec id 3, properties len 49
[20220923-11:34:44] [INFO ] Loading keymap file /etc/xrdp/km-00000409.ini
[20220923-11:34:44] [WARN ] local keymap file for 0x00000409 found and doesn't match built in keymap, using local keymap file
[20220923-11:34:48] [WARN ] xrdp_wm_log_msg: 'ip' is not needed for this connection - please remove
[20220923-11:34:48] [INFO ] 'ip' is not needed for this connection - please remove
[20220923-11:34:48] [INFO ] connecting to sesman on sesman.socket
[20220923-11:34:48] [WARN ] Ignoring obsolete SCP port value '3350'
[20220923-11:34:48] [INFO ] xrdp_wm_log_msg: sesman connect ok
[20220923-11:34:48] [INFO ] sesman connect ok
[20220923-11:34:48] [INFO ] sending login info to session manager. Please wait...
[20220923-11:34:49] [INFO ] xrdp_wm_log_msg: login successful for user noorez on display 10
[20220923-11:34:49] [INFO ] login successful for user noorez on display 10
[20220923-11:34:49] [INFO ] loaded module 'libxup.so' ok, interface size 10408, version 4
[20220923-11:34:49] [INFO ] started connecting
[20220923-11:34:49] [INFO ] lib_mod_connect: connecting via UNIX socket
[20220923-11:34:51] [INFO ] lib_mod_log_peer: xrdp_pid=3668 connected to X11rdp_pid=3673 X11rdp_uid=1000 X11rdp_gid=1000 client=AF_VSOCK:cid=2/port=2261028321
[20220923-11:34:51] [INFO ] connected ok
[20220923-11:38:05] [INFO ] Received termination signal, stopping the server accept new connections thread
[20220923-11:39:01] [INFO ] address [-1] port [3389] mode 3
[20220923-11:39:01] [INFO ] listening to port 3389 on -1
[20220923-11:39:01] [INFO ] xrdp_listen_pp done
[20220923-11:39:03] [INFO ] starting xrdp with pid 800
[20220923-11:39:03] [INFO ] address [-1] port [3389] mode 3
[20220923-11:39:03] [INFO ] listening to port 3389 on -1
[20220923-11:39:03] [INFO ] xrdp_listen_pp done
[20220923-11:40:43] [INFO ] Socket 12: connection accepted from AF_VSOCK:cid=2/port=2261028426
[20220923-11:40:43] [INFO ] Security protocol: configured [RDP], requested [SSL|HYBRID|HYBRID_EX|RDP], selected [RDP]
[20220923-11:40:43] [ERROR] libxrdp_force_read: header read error
[20220923-11:40:43] [ERROR] Processing [ITU-T T.125] Connect-Initial failed
[20220923-11:40:43] [ERROR] [MCS Connection Sequence] receive connection request failed
[20220923-11:40:43] [INFO ] Socket 12: connection accepted from AF_VSOCK:cid=2/port=2261028427
[20220923-11:40:43] [ERROR] xrdp_sec_incoming: xrdp_mcs_incoming failed
[20220923-11:40:43] [INFO ] Security protocol: configured [RDP], requested [RDP], selected [RDP]
[20220923-11:40:43] [ERROR] xrdp_rdp_incoming: xrdp_sec_incoming failed
[20220923-11:40:43] [INFO ] Connected client computer name: EDNOOREZK
[20220923-11:40:43] [ERROR] xrdp_process_main_loop: libxrdp_process_incoming failed
[20220923-11:40:43] [WARN ] Client and server do not both support the same encryption.
[20220923-11:40:43] [ERROR] xrdp_iso_send: trans_write_copy_s failed
[20220923-11:40:43] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc006 is unknown (ignored)
[20220923-11:40:43] [ERROR] Sending [ITU T.125] DisconnectProviderUltimatum failed
[20220923-11:40:43] [WARN ] Received [MS-RDPBCGR] TS_UD_HEADER type 0xc00a is unknown (ignored)
[20220923-11:40:43] [INFO ] xrdp_load_keyboard_layout: Keyboard information sent by the RDP client, keyboard_type:[0x04], keyboard_subtype:[0x00], keylayout:[0x00000409]
[20220923-11:40:43] [INFO ] xrdp_load_keyboard_layout: model [] variant [] layout [us] options []
[20220923-11:40:43] [INFO ] Non-TLS connection established from AF_VSOCK:cid=2/port=2261028427 with security level : none
[20220923-11:40:44] [INFO ] xrdp_caps_process_pointer: client supports new(color) cursor
[20220923-11:40:44] [INFO ] xrdp_process_offscreen_bmpcache: support level 1 cache size 10485760 MB cache entries 100
[20220923-11:40:44] [INFO ] xrdp_caps_process_codecs: nscodec, codec id 1, properties len 3
[20220923-11:40:44] [WARN ] xrdp_caps_process_codecs: unknown codec id 5
[20220923-11:40:44] [INFO ] xrdp_caps_process_codecs: RemoteFX, codec id 3, properties len 49
[20220923-11:40:44] [INFO ] Loading keymap file /etc/xrdp/km-00000409.ini
[20220923-11:40:44] [WARN ] local keymap file for 0x00000409 found and doesn't match built in keymap, using local keymap file
[20220923-11:40:48] [WARN ] xrdp_wm_log_msg: 'ip' is not needed for this connection - please remove
[20220923-11:40:48] [INFO ] 'ip' is not needed for this connection - please remove
[20220923-11:40:48] [INFO ] connecting to sesman on sesman.socket
[20220923-11:40:48] [WARN ] Ignoring obsolete SCP port value '3350'
[20220923-11:40:48] [INFO ] xrdp_wm_log_msg: sesman connect ok
[20220923-11:40:48] [INFO ] sesman connect ok
[20220923-11:40:48] [INFO ] sending login info to session manager. Please wait...
[20220923-11:40:48] [INFO ] xrdp_wm_log_msg: login successful for user noorez on display 10
[20220923-11:40:48] [INFO ] login successful for user noorez on display 10
[20220923-11:40:48] [INFO ] loaded module 'libxup.so' ok, interface size 10408, version 4
[20220923-11:40:48] [INFO ] started connecting
[20220923-11:40:48] [INFO ] lib_mod_connect: connecting via UNIX socket
[20220923-11:40:50] [INFO ] lib_mod_log_peer: xrdp_pid=1568 connected to X11rdp_pid=1586 X11rdp_uid=1000 X11rdp_gid=1000 client=AF_VSOCK:cid=2/port=2261028427
[20220923-11:40:50] [INFO ] connected ok

copy from /var/log/xrdp-sesman.log

[20220923-11:32:38] [INFO ] Calling auth_stop_session and auth_end from pid 2676
[20220923-11:32:38] [INFO ] Terminating X server (pid 2678) on display 10
[20220923-11:32:38] [INFO ] Terminating the xrdp channel server (pid 2686) on display 10
[20220923-11:32:38] [INFO ] X server on display 10 (pid 2678) returned exit code 0 and signal number 0
[20220923-11:32:38] [INFO ] xrdp channel server for display 10 (pid 2686) exit code 0 and signal number 0
[20220923-11:32:38] [INFO ] cleanup_sockets:
[20220923-11:32:38] [INFO ] Process 2676 has exited
[20220923-11:32:38] [INFO ] ++ terminated session:  username noorez, display :10.0, session_pid 2676, ip 
[20220923-11:32:59] [INFO ] sesman_main_loop: sesman asked to terminate
[20220923-11:33:16] [INFO ] starting xrdp-sesman with pid 3577
[20220923-11:33:16] [INFO ] Sesman now listening on /var/run/xrdp-sesman/sesman.socket
[20220923-11:33:16] [INFO ] sesman_main_loop: sesman asked to terminate
[20220923-11:33:38] [INFO ] starting xrdp-sesman with pid 3602
[20220923-11:33:38] [INFO ] Sesman now listening on /var/run/xrdp-sesman/sesman.socket
[20220923-11:33:38] [INFO ] sesman_main_loop: sesman asked to terminate
[20220923-11:34:37] [INFO ] starting xrdp-sesman with pid 3652
[20220923-11:34:37] [INFO ] Sesman now listening on /var/run/xrdp-sesman/sesman.socket
[20220923-11:34:48] [INFO ] Socket 13: connection accepted from AF_UNIX
[20220923-11:34:49] [INFO ] Received request to create Xorg session for user: noorez
[20220923-11:34:49] [INFO ] Terminal Server Users group is disabled, allowing authentication
[20220923-11:34:49] [INFO ] ++ created session (access granted): username noorez
[20220923-11:34:49] [INFO ] Starting session: session_pid 3671, display :10.0, width 3840, height 2160, bpp 24, client ip , user name noorez
[20220923-11:34:49] [INFO ] [session start] (display 10): calling auth_start_session from pid 3671
[20220923-11:34:49] [INFO ] Starting X server on display 10: Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log  
[20220923-11:34:49] [INFO ] Found X server running at /tmp/.X11-unix/X10
[20220923-11:34:49] [INFO ] Found X server running at /tmp/.X11-unix/X10
[20220923-11:34:49] [INFO ] Session started successfully for user noorez on display 10
[20220923-11:34:49] [INFO ] Starting the xrdp channel server for display 10
[20220923-11:34:49] [INFO ] Found X server running at /tmp/.X11-unix/X10
[20220923-11:34:49] [INFO ] Session in progress on display 10, waiting until the window manager (pid 3672) exits to end the session
[20220923-11:34:49] [INFO ] Starting the default window manager on display 10: /etc/xrdp/startubuntu.sh
[20220923-11:34:51] [ERROR] sesman_main_loop: trans_check_wait_objs failed, removing trans
[20220923-11:38:05] [INFO ] sesman_main_loop: sesman asked to terminate
[20220923-11:38:54] [WARN ] Window manager (pid 3672, display 10) exited with non-zero exit code 255 and signal 1. This could indicate a window manager config problem
[20220923-11:38:54] [INFO ] Calling auth_stop_session and auth_end from pid 3671
[20220923-11:38:54] [INFO ] Terminating X server (pid 3673) on display 10
[20220923-11:38:54] [INFO ] Terminating the xrdp channel server (pid 3681) on display 10
[20220923-11:38:54] [INFO ] X server on display 10 (pid 3673) returned exit code 0 and signal number 0
[20220923-11:38:54] [INFO ] xrdp channel server for display 10 (pid 3681) exit code 255 and signal number 1
[20220923-11:38:54] [INFO ] cleanup_sockets:
[20220923-11:39:01] [INFO ] starting xrdp-sesman with pid 765
[20220923-11:39:01] [INFO ] Sesman now listening on /var/run/xrdp-sesman/sesman.socket
[20220923-11:40:48] [INFO ] Socket 13: connection accepted from AF_UNIX
[20220923-11:40:48] [INFO ] Received request to create Xorg session for user: noorez
[20220923-11:40:48] [INFO ] Terminal Server Users group is disabled, allowing authentication
[20220923-11:40:48] [INFO ] ++ created session (access granted): username noorez
[20220923-11:40:48] [INFO ] Starting session: session_pid 1571, display :10.0, width 3840, height 2160, bpp 24, client ip , user name noorez
[20220923-11:40:48] [INFO ] [session start] (display 10): calling auth_start_session from pid 1571
[20220923-11:40:49] [INFO ] Starting X server on display 10: Xorg :10 -auth .Xauthority -config xrdp/xorg.conf -noreset -nolisten tcp -logfile .xorgxrdp.%s.log  
[20220923-11:40:49] [INFO ] Found X server running at /tmp/.X11-unix/X10
[20220923-11:40:49] [INFO ] Found X server running at /tmp/.X11-unix/X10
[20220923-11:40:49] [INFO ] Session started successfully for user noorez on display 10
[20220923-11:40:49] [INFO ] Found X server running at /tmp/.X11-unix/X10
[20220923-11:40:49] [INFO ] Starting the default window manager on display 10: /etc/xrdp/startubuntu.sh
[20220923-11:40:49] [INFO ] Starting the xrdp channel server for display 10
[20220923-11:40:49] [INFO ] Session in progress on display 10, waiting until the window manager (pid 1585) exits to end the session
[20220923-11:40:50] [ERROR] sesman_main_loop: trans_check_wait_objs failed, removing trans

noorez avatar Sep 23 '22 17:09 noorez

I'll need to set up a VM to look at the crash. It'll take me a while.

One thing is currently confusing me - according to the crash file the executable is dated Thu 18 Aug 23:32:42 BST 2022. The date of the executable from the deb file appears to be Wed 16 Feb 17:38:50 GMT 2022.

Can you give me the output of the following two commands:-

ls -l /usr/sbin/xrdp
sha256sum /usr/sbin/xrdp

Thanks.

matt335672 avatar Oct 10 '22 18:10 matt335672

It seems I have lost the /var/crash directory ... :( I will wait for the same crash message again and get the details you need for it.

On Mon., Oct. 10, 2022, 12:40 matt335672, @.***> wrote:

I'll need to set up a VM to look at the crash. It'll take me a while.

One thing is currently confusing me - according to the crash file the executable is dated Thu 18 Aug 23:32:42 BST 2022. The date of the executable from the deb file appears to be Wed 16 Feb 17:38:50 GMT 2022.

Can you give me the output of the following two commands:-

ls -l /usr/sbin/xrdp sha256sum /usr/sbin/xrdp

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/neutrinolabs/xrdp/issues/2377#issuecomment-1273684427, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAY546576JNUYCR2A7KFOWDWCRPKRANCNFSM6AAAAAAQRVMDKU . You are receiving this because you authored the thread.Message ID: @.***>

noorez avatar Oct 10 '22 19:10 noorez

I've got the crash, so that shouldn't be necessary. The outputs of those other two commands could be useful though.

matt335672 avatar Oct 10 '22 20:10 matt335672 

$ ls -l /usr/sbin/xrdp
-rwxr-xr-x 1 root root 190696 Aug 18 16:32 /usr/sbin/xrdp
$ sha256sum /usr/sbin/xrdp
cc38f5c18a496825748962ca98af9c1fb4d3732b6d23de0aa5499010ae4e3e93  /usr/sbin/xrdp

perhaps the confusion in dates is happening is because I updated the debian package file manually to point so that I could get the latest in origin/devel rather than what is currently maintained by ubuntu package maintainers?

noorez avatar Oct 11 '22 15:10 noorez

@norez - it seems likely you've found a problem with the wavefront you've build from origin/devel. That's fine, but to debug it we need the debugging symbols corresponding to your private build.

Have you built your own .deb file? If so, have you also got a .ddeb file and can you tell me the commit you built? I'm assuming you haven't supplied your own private patches.

matt335672 avatar Oct 12 '22 09:10 matt335672

77a8b643790fb5ef974f0833c8a6ede94af3e44e is the commit hash I was on when I built

I uploaded the debug symbols and also the debian patch files I carried over for systemd and permissions to work.

xrdp_2377.zip

EDIT: since xorgxrdp was built as a separate .deb 715b430bcd28d6aa38f32945f82353b3273b1213 is the commit for that repository as well

noorez avatar Oct 12 '22 13:10 noorez

Thanks.

Are you able to post /usr/sbin/xrdp too? I can try building it here, but I'll be lucky to get an exact match.

matt335672 avatar Oct 13 '22 11:10 matt335672

I updated the archive wtih my .deb files that I built. This should hopefully get you that match you need. I built them for Ubuntu 22.04 (jammy)

xrdp_2377.zip

noorez avatar Oct 13 '22 12:10 noorez

That's great - thanks.

The stack trace from your coredump is:-

#0  0x000056090a517776 in loop1f ()
#1  0x000056090a517984 in rfxcodec_encode_dwt_shift_amd64_sse41 ()
#2  0x000056090a51af48 in __func__.0 ()
#3  0x00007f06b9d8b000 in ?? ()
#4  0x000056090d1e3eb0 in ?? ()
#5  0x000056090d1e5eb0 in ?? ()
#6  0x000056090d1dfe60 in ?? ()
#7  0x000056090a5092e7 in rfx_encode_component_rlgr3_amd64_sse41 (enc=0x56090d1dfe60, qtable=<optimised out>, data=<optimised out>, 
    buffer=0x7f06b400ad2b "Z41EJ\251\234\271\030\t%Сh\345]\247k\237\234\003\261<\243\"C\316<\354+E+\350\350C\260!N$%\n@$\224\253\255\vw5\202D\241D\242\374u\203\021\tI\225DEg#\025\241(\301(\032B\264\201\254\245\020R\244\034H\203D\024$\200D\232\024\202BH4\030\"Q)rb\a(\361\343\025\001\250e\f0AJ\030\250`\324Q!a'\004P\240\232\024\026J\002\206\211\250\240\224\003j\v44\361,8 \264\024\342\206\v\023\t\266,\004\345\350\004ȕA\340|\322A#\213\f<\301\340\204\026I\314G\204", buffer_size=33398661, size=0x7f06bb5cab8c)
    at ../librfxcodec/src/amd64/rfxencode_tile_amd64.c:99
#8  0x000056090a4f9a1f in check_and_rfx_encode (size=0x7f06bb5cab8c, buffer_size=33398661, 
    buffer=0x7f06b400ad2b "Z41EJ\251\234\271\030\t%Сh\345]\247k\237\234\003\261<\243\"C\316<\354+E+\350\350C\260!N$%\n@$\224\253\255\vw5\202D\241D\242\374u\203\021\tI\225DEg#\025\241(\301(\032B\264\201\254\245\020R\244\034H\203D\024$\200D\232\024\202BH4\030\"Q)rb\a(\361\343\025\001\250e\f0AJ\030\250`\324Q!a'\004P\240\232\024\026J\002\206\211\250\240\224\003j\v44\361,8 \264\024\342\206\v\023\t\266,\004\345\350\004ȕA\340|\322A#\213\f<\301\340\204\026I\314G\204", data=<optimised out>, 
    qtable=<optimised out>, enc=<optimised out>) at ../librfxcodec/src/rfxencode_tile.c:130
#9  check_and_rfx_encode (size=0x7f06bb5cab8c, buffer_size=33398661, 
    buffer=0x7f06b400ad2b "Z41EJ\251\234\271\030\t%Сh\345]\247k\237\234\003\261<\243\"C\316<\354+E+\350\350C\260!N$%\n@$\224\253\255\vw5\202D\241D\242\374u\203\021\tI\225DEg#\025\241(\301(\032B\264\201\254\245\020R\244\034H\203D\024$\200D\232\024\202BH4\030\"Q)rb\a(\361\343\025\001\250e\f0AJ\030\250`\324Q!a'\004P\240\232\024\026J\002\206\211\250\240\224\003j\v44\361,8 \264\024\342\206\v\023\t\266,\004\345\350\004ȕA\340|\322A#\213\f<\301\340\204\026I\314G\204", data=<optimised out>, 
    qtable=<optimised out>, enc=<optimised out>) at ../librfxcodec/src/rfxencode_tile.c:122
#10 rfx_encode_yuv (width=<optimised out>, height=<optimised out>, stride_bytes=<optimised out>, v_size=0x7f06bb5cab94, u_size=0x7f06bb5cab90, y_size=0x7f06bb5cab8c, 
    data_out=<synthetic pointer>, v_quants=0x56090a51af48 <g_rfx_default_quantization_values> "ffw\210\230", 
    u_quants=0x56090a51af48 <g_rfx_default_quantization_values> "ffw\210\230", y_quants=<optimised out>, yuv_data=<optimised out>, enc=<optimised out>)
    at ../librfxcodec/src/rfxencode_tile.c:254
#11 rfx_compose_message_tile_yuv (tile_width=<optimised out>, tile_height=<optimised out>, stride_bytes=<optimised out>, yIdx=<optimised out>, xIdx=<optimised out>, 
    quantIdxCr=0, quantIdxCb=0, quantIdxY=<optimised out>, quantVals=<optimised out>, tile_data=<optimised out>, s=<synthetic pointer>, enc=<optimised out>)
    at ../librfxcodec/src/rfxencode_compose.c:239
#12 rfx_compose_message_tileset (quants=0x0, num_quants=0, flags=0, width=<optimised out>, height=<optimised out>, num_tiles=2040, tiles=0x7f06b5fe4cb0, 
    stride_bytes=<optimised out>, buf=0x7f06b8deb000 <error: Cannot access memory at address 0x7f06b8deb000>, s=<synthetic pointer>, enc=<optimised out>)
    at ../librfxcodec/src/rfxencode_compose.c:504
#13 rfx_compose_message_data (width=<optimised out>, height=<optimised out>, quants=0x0, num_quants=0, flags=0, num_tiles=2040, tiles=0x7f06b5fe4cb0, 
    stride_bytes=<optimised out>, buf=0x7f06b8deb000 <error: Cannot access memory at address 0x7f06b8deb000>, num_regions=<optimised out>, regions=0x7f06b5ff2bd0, 
    s=<synthetic pointer>, enc=<optimised out>) at ../librfxcodec/src/rfxencode_compose.c:614
#14 rfxcodec_encode_ex (width=<optimised out>, height=<optimised out>, quants=0x0, num_quants=0, flags=0, num_tiles=2040, tiles=0x7f06b5fe4cb0, 
    num_regions=<optimised out>, regions=0x7f06b5ff2bd0, stride_bytes=<optimised out>, buf=0x7f06b8deb000 <error: Cannot access memory at address 0x7f06b8deb000>, 
    cdata_bytes=<synthetic pointer>, cdata=0x7f06b4000cb0 "\304\314\016", handle=<optimised out>) at ../librfxcodec/src/rfxencode.c:373
#15 rfxcodec_encode (width=<optimised out>, height=<optimised out>, quants=0x0, num_quants=0, num_tiles=2040, tiles=0x7f06b5fe4cb0, num_regions=<optimised out>, 
    regions=0x7f06b5ff2bd0, stride_bytes=<optimised out>, buf=0x7f06b8deb000 <error: Cannot access memory at address 0x7f06b8deb000>, cdata_bytes=<synthetic pointer>, 
    cdata=0x7f06b4000cb0 "\304\314\016", handle=<optimised out>) at ../librfxcodec/src/rfxencode.c:389
#16 process_enc_rfx (self=0x56090ad4ba60, enc=0x56090d2a5fb0) at ./xrdp/xrdp_encoder.c:391
#17 0x000056090a4f9d03 in proc_enc_msg (arg=0x56090ad4ba60) at ./xrdp/xrdp_encoder.c:527
#18 0x000056090a4f9d52 in proc_enc_msg (arg=<optimised out>) at ./xrdp/xrdp_encoder.c:456
#19 0x00007f06bca0ab43 in start_thread (arg=<optimised out>) at ./nptl/pthread_create.c:442
#20 0x00007f06bca9ca00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

Essentially there's a separate thread that runs to encode the frames that are sent from the server to the client when using RFX. The code is quite heavily optimised, and the functions at the top of the trace are in x86_64 assembler.

When xrdp shuts down, the thread needs to be terminated. It's not my area, but it looks like the shared memory used by the separate thread has been deleted before the thread has finished, and this is causing the coredump.

We've been refactoring this area quite a bit as part of development, so this kind of thing is not entirely unexpected. Since xrdp is in the process of quitting anyway this is relatively benign. We've made no other changes to this area since 77a8b643790fb5ef974f0833c8a6ede94af3e44e, so updating to a later development build won't help here.

@Nexarian - could you have a quick look at the above and let me know if there's anything you could do with from the coredump?

matt335672 avatar Oct 14 '22 10:10 matt335672

Wondering if there are any updates on this I could try?

noorez avatar Dec 14 '22 13:12 noorez

Nothing yet.

matt335672 avatar Dec 15 '22 09:12 matt335672

I got here because I searched for an error that I encountered testing the new nvidia acceleration in devel build. The error is Physical desktop dimensions (0x0) are invalid.

Isn't this a relevant error from the original post?

If invalid dimensions are passed along unhandled, this will most likely result in a crash.

Since this issue dates from before two weeks ago, I can safely assume this error message isn't related to nvidia support. Although xrdp installed from apt works just fine.

Quedale avatar Feb 12 '24 17:02 Quedale

This can be closed now. With the GFX implementation we are using shared memory in a totally different way.

matt335672 avatar Mar 22 '24 17:03 matt335672