rr icon indicating copy to clipboard operation
rr copied to clipboard

`rr replay` fails with "Unknown debugger request 35"

Open froydnj opened this issue 3 years ago • 11 comments

Freshly built gdb from 8e2de6c81a90e98ed95dfc0ffef1e3df9cfff6b7:

froydnj@pyro:~/src/sorbet$ ../rr/bin/rr record bazel-bin/main/sorbet-orig --silence-dev-message --store-state bazel-out/k8-dbg/bin/payload/binary/state-payload-raw --no-error-count                              
rr: Saving execution to trace directory `/home/froydnj/.local/share/rr/sorbet-orig-3'.
Illegal instruction
froydnj@pyro:~/src/sorbet$ ../rr/bin/rr replay -d ~/gdb/bin/gdb
GNU gdb (GDB) 13.0.50.20220614-git
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/froydnj/.local/share/rr/sorbet-orig-3/mmap_hardlink_3_sorbet-orig...
Really redefine built-in command "restart"? (y or n) [answered Y; input not from terminal]
Really redefine built-in command "jump"? (y or n) [answered Y; input not from terminal]
Remote debugging using 127.0.0.1:9092
Reading symbols from /lib64/ld-linux-x86-64.so.2...
(No debugging symbols found in /lib64/ld-linux-x86-64.so.2)
BFD: warning: system-supplied DSO at 0x6fffd000 has a section extending past end of file
[FATAL /home/froydnj/src/rr/src/GdbServer.cc:806:dispatch_debugger_request()] Unknown debugger request 35
=== Start rr backtrace:
../rr/bin/rr(_ZN2rr13dump_rr_stackEv+0x35)[0x5588a2618042]
../rr/bin/rr(_ZN2rr15notifying_abortEv+0x12)[0x5588a2617fc9]
../rr/bin/rr(_ZN2rr12FatalOstreamD1Ev+0x30)[0x5588a2482cd2]
../rr/bin/rr(_ZN2rr9GdbServer25dispatch_debugger_requestERNS_7SessionERKNS_10GdbRequestENS0_11ReportStateE+0x24aa)[0x5588a244ab9c]
../rr/bin/rr(_ZN2rr9GdbServer25process_debugger_requestsENS0_11ReportStateE+0x42f)[0x5588a244ce9b]
../rr/bin/rr(_ZN2rr9GdbServer14debug_one_stepERNS_10GdbRequestE+0x172)[0x5588a244d9dc]
../rr/bin/rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x52c)[0x5588a24502bc]
../rr/bin/rr(+0x48b9a4)[0x5588a25619a4]
../rr/bin/rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x3cd)[0x5588a2562357]
../rr/bin/rr(main+0x26c)[0x5588a263266b]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f7fc50902e1]
../rr/bin/rr(_start+0x2a)[0x5588a23b0d5a]
=== End rr backtrace
Remote connection closed

...followed by GDB falling over, which I assume is not the real problem here.

The same thing happens with GDB 7.12 (yes, yes, ancient GDB...).

Let me know if you need anything else.

froydnj avatar Jun 14 '22 20:06 froydnj

DREQ 35 is DREQ_QSYMBOL, the processing for which is missing if PROC_SERVICE_H is not defined (https://github.com/rr-debugger/rr/blob/86655e3eec738c66ae51c03f5ccd04199ef561dd/src/GdbServer.cc#L750), so that might suggest a workaround (find out what package provides the header and install it). That said, we should either make it mandatory or have an appropriate fallback.

Keno avatar Jun 14 '22 20:06 Keno

Yeah does your system have a header that defines ps_get_thread_area?

khuey avatar Jun 14 '22 21:06 khuey

Yeah does your system have a header that defines ps_get_thread_area?

I have syscall definitions for __NR_get_thread_area and friends, but ps_get_thread_area is nowhere to be found; my glibc must not be new enough?

froydnj avatar Jun 15 '22 01:06 froydnj

What distro is this?

khuey avatar Jun 15 '22 01:06 khuey

What distro is this?

Debian stretch.

froydnj avatar Jun 15 '22 01:06 froydnj

Do you have libc6-dev installed?

khuey avatar Jun 15 '22 01:06 khuey

Do you have libc6-dev installed?

Yes.

froydnj avatar Jun 15 '22 01:06 froydnj

https://packages.debian.org/search?suite=buster&arch=any&mode=filename&searchon=contents&keywords=proc_service vs https://packages.debian.org/search?suite=stretch&arch=any&mode=filename&searchon=contents&keywords=proc_service seems like a Debian packaging error tbh.

khuey avatar Jun 15 '22 01:06 khuey

That's unfortunate. I guess I should go grab proc_service.h out of glibc's git and stick it in the correct location.

...though it looks like the relevant shared libraries don't even have the symbol in question, so just having the header wouldn't help very much.

froydnj avatar Jun 15 '22 01:06 froydnj

That said, we should at least not die here, so I pushed c15a5b9b4af477f756d5e9f01250dd043a7654d2

Not sure how gdb will react to that but rr shouldn't crash.

khuey avatar Jun 15 '22 01:06 khuey

Do you have a libthread_db.so on your file system by chance?

khuey avatar Jun 15 '22 01:06 khuey