flycast
flycast copied to clipboard
flyinghead
GDB step instruction crashing the emulator
Please Note: This form is the minimum required information for submitting bugs.
Removing this form may lead to your issue being closed until it is completed.
Platform / OS / Hardware: Windows 11 24H2
Flycast version: latest master
Hardware: N/A
Description of the Issue
A GDB step will trigger the condition where the verify will encounter a CpuRunning at true and crash the whole emulator.
Debugging Steps Tested
- Debug flycast itself
- I'm using dcload-ip (sizious version) to test it
- Connect GDB
- Optionally add a breakpoint in the DC exec
- Let the emulated app run
- Stop the emulated app or wait for a breakpoint
- Try to step
- It should immediately crash
Logs Gathered
00:19:667 network\bridge.cpp:128 W[NETWORK]: Processing packet
00:22:344 hw\sh4\interpr\sh4_opcodes.cpp:929 W[INTERPRETER]: TRAP #8
Exception thrown at 0x00007FF9D209804A in flycast.exe: Microsoft C++ exception: debugger::Stop at memory location 0x000000535ADFEC90.
01:03:865 ui\gui.cpp:1568 E[COMMON]: Verify Failed : !ctx->CpuRunning
in Sh4Interpreter::Step -> C:\Users\arnod\Code\flycast\core\hw\sh4\interpr\sh4_interpreter.cpp : 83
Debug Error!
Program: C:\Users\arnod\Code\flycast\build\Debug\flycast.exe
abort() has been called
(Press Retry to debug the application)
Screenshots
I can't seem to reproduce this one. Here is an example gdb session that should reproduce the issue but doesn't:
Remote debugging using localhost:3263
warning: No executable has been specified and target does not support
determining executable automatically. Try using the "file" command.
0x8c0335ba in ?? ()
(gdb) br *0x8c0335ba
Breakpoint 1 at 0x8c0335ba
(gdb) c
Continuing.
Breakpoint 1, 0x8c0335ba in ?? ()
(gdb) si
0x8c069a94 in ?? ()
(gdb) si
0x8c069a96 in ?? ()
(gdb) si
0x8c069a98 in ?? ()
(gdb) c
Continuing.
Breakpoint 1, 0x8c0335ba in ?? ()
(gdb) si
0x8c069ad4 in ?? ()
Is there anything I'm missing? I noticed that the debugger is seriously broken when disabling multi-threading (Settings > Advanced) so I assume you have it on. Any other non-default setting?
So, the instruction that was sent by GDB precisely is: $vCont;r8c009530,8c009532:0;c#85\n (I'm using the vscode GUI to debug the program)
Interestingly enough, even if I'm doing multiple sis, I'm crashing eventually:
GDB logs
GNU gdb (Debian 16.3-1) 16.3
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
The target architecture is set to "sh4".
The target is set to little endian.
[remote] start_remote_1: enter
[remote] Sending packet: $qSupported:multiprocess+;swbreak+;hwbreak+;qRelocInsn+;fork-events+;vfork-events+;exec-events+;vContSupported+;QThreadEvents+;QThreadOptions+;no-resumed+;memory-tagging+;xmlRegisters=i386;error-message+#14
[remote] Received Ack
[remote] Packet received: PacketSize=4096;vContSupported+
[remote] packet_ok: Packet qSupported (supported-packets) is supported
[remote] Sending packet: $vCont?#49
[remote] Received Ack
[remote] Packet received: vCont;c;C;s;S;t;r
[remote] packet_ok: Packet vCont (verbose-resume) is supported
[remote] Sending packet: $vMustReplyEmpty#3a
[remote] Received Ack
[remote] Packet received:
[remote] Sending packet: $Hg0#df
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $qTStatus#49
[remote] Received Ack
[remote] Packet received:
[remote] packet_ok: Packet qTStatus (trace-status) is NOT supported
[remote] Sending packet: $?#3f
[remote] Received Ack
[remote] Packet received: S02
[remote] Sending packet: $qfThreadInfo#bb
[remote] Received Ack
[remote] Packet received: m0
[remote] Sending packet: $qsThreadInfo#c8
[remote] Received Ack
[remote] Packet received: l
[remote] Sending packet: $qAttached#8f
[remote] Received Ack
[remote] Packet received: 1
[remote] packet_ok: Packet qAttached (query-attached) is supported
[remote] Sending packet: $Hc-1#09
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $qC#b4
[remote] Received Ack
[remote] Packet received: QC0.01
[remote] remote_current_thread: warning: garbage in qC reply
[remote] Sending packet: $qOffsets#4b
[remote] Received Ack
[remote] Packet received:
[remote] wait: enter
[remote] select_thread_for_ambiguous_stop_reply: enter
[remote] select_thread_for_ambiguous_stop_reply: process_wide_stop = 0
[remote] select_thread_for_ambiguous_stop_reply: first resumed thread is Remote target
[remote] select_thread_for_ambiguous_stop_reply: is this guess ambiguous? = 0
[remote] select_thread_for_ambiguous_stop_reply: exit
[remote] wait: exit
[remote] Sending packet: $g#67
[remote] Received Ack
[remote] Packet received: 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000f0000070000000000100040000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
[remote] Sending packet: $qfThreadInfo#bb
[remote] Received Ack
[remote] Packet received: m0
[remote] Sending packet: $qsThreadInfo#c8
[remote] Received Ack
[remote] Packet received: l
[remote] Sending packet: $ma0000000,2#7c
[remote] Received Ack
[remote] Packet received: 5b08
[remote] Sending packet: $m9ffffffe,2#cd
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffffc,2#cb
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffffa,2#c9
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff8,2#a0
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff6,2#9e
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff4,2#9c
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff2,2#9a
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $ma0000000,2#7c
[remote] Received Ack
[remote] Packet received: 5b08
[remote] Sending packet: $m9ffffffe,2#cd
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffffc,2#cb
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffffa,2#c9
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff8,2#a0
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff6,2#9e
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff4,2#9c
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $m9ffffff2,2#9a
[remote] Received Ack
[remote] Packet received: 0000
0xa0000000 in ?? ()
[remote] Sending packet: $qSymbol::#5b
[remote] Received Ack
[remote] Packet received: OK
[remote] packet_ok: Packet qSymbol (symbol-lookup) is supported
[remote] start_remote_1: exit
[remote] Sending packet: $m8c0058b2,2#c7
[remote] Received Ack
[remote] Packet received: 0000
Breakpoint 1 at 0x8c0058b2: file rtl8139.c, line 783.
[remote] Sending packet: $m8c005a00,2#bc
[remote] Received Ack
[remote] Packet received: 0000
Breakpoint 2 at 0x8c005a00: file rtl8139.c, line 900.
(gdb) si
[remote] Sending packet: $Z0,8c0058b2,2#10
[remote] Received Ack
[remote] Packet received: OK
[remote] packet_ok: Packet Z0 (software-breakpoint) is supported
[remote] Sending packet: $Z0,8c005a00,2#05
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $vCont;s:0;c#c0
[remote] Received Ack
[remote] wait: enter
[remote] Packet received: S05
[remote] select_thread_for_ambiguous_stop_reply: enter
[remote] select_thread_for_ambiguous_stop_reply: process_wide_stop = 0
[remote] select_thread_for_ambiguous_stop_reply: first resumed thread is Remote target
[remote] select_thread_for_ambiguous_stop_reply: is this guess ambiguous? = 0
[remote] select_thread_for_ambiguous_stop_reply: exit
[remote] wait: exit
[remote] Sending packet: $g#67
[remote] Received Ack
[remote] Packet received: 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008d008300ac3c0400ac0000008c0000008c0000000000000000f1000040000000000100040000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000407607008c00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f2,2#f1
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f2,2#f1
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $qfThreadInfo#bb
[remote] Received Ack
[remote] Packet received: m0
[remote] Sending packet: $qsThreadInfo#c8
[remote] Received Ack
[remote] Packet received: l
[remote] Sending packet: $z0,8c0058b2,2#30
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $z0,8c005a00,2#25
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f2,2#f1
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f2,2#f1
[remote] Received Ack
[remote] Packet received: 0000
0xac008300 in ?? ()
(gdb) s
Cannot find bounds of current function
(gdb) s
Cannot find bounds of current function
(gdb) si
[remote] Sending packet: $Z0,8c0058b2,2#10
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $Z0,8c005a00,2#05
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $vCont;s:0;c#c0
[remote] Received Ack
[remote] wait: enter
[remote] Packet received: S05
[remote] select_thread_for_ambiguous_stop_reply: enter
[remote] select_thread_for_ambiguous_stop_reply: process_wide_stop = 0
[remote] select_thread_for_ambiguous_stop_reply: first resumed thread is Remote target
[remote] select_thread_for_ambiguous_stop_reply: is this guess ambiguous? = 0
[remote] select_thread_for_ambiguous_stop_reply: exit
[remote] wait: exit
[remote] Sending packet: $g#67
[remote] Received Ack
[remote] Packet received: 000000ff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008d028300ac3c0400ac0000008c0000008c0000000000000000f1000040000000000100040000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000407607008c000000ff000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
[remote] Sending packet: $mac008302,2#bc
[remote] Received Ack
[remote] Packet received: 08d1
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac008302,2#bc
[remote] Received Ack
[remote] Packet received: 08d1
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $qfThreadInfo#bb
[remote] Received Ack
[remote] Packet received: m0
[remote] Sending packet: $qsThreadInfo#c8
[remote] Received Ack
[remote] Packet received: l
[remote] Sending packet: $z0,8c0058b2,2#30
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $z0,8c005a00,2#25
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $mac008302,2#bc
[remote] Received Ack
[remote] Packet received: 08d1
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac008302,2#bc
[remote] Received Ack
[remote] Packet received: 08d1
[remote] Sending packet: $mac008300,2#ba
[remote] Received Ack
[remote] Packet received: 07d0
[remote] Sending packet: $mac0082fe,2#24
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fc,2#22
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082fa,2#20
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f8,2#f7
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f6,2#f5
[remote] Received Ack
[remote] Packet received: 0000
[remote] Sending packet: $mac0082f4,2#f3
[remote] Received Ack
[remote] Packet received: 0000
0xac008302 in ?? ()
(gdb) si
[remote] Sending packet: $Z0,8c0058b2,2#10
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $Z0,8c005a00,2#05
[remote] Received Ack
[remote] Packet received: OK
[remote] Sending packet: $vCont;s:0;c#c0
**emulator crash, same way as before**
The binary I'm trying to launch (as-is) is: 1st_read.zip
(it's basically the master of https://github.com/sizious/dcload-ip, compiled by removing the -Os flags and putting -Og -ggdb in the Makefile)
I reset the settings first to make sure I did not miss anything. Multithreading is indeed enabled, I disabled UPNP, enabled the BBA emulation, enabled interpreter and GDB + wait for connect. Also tried to disable BBA, but nothing :/
I still can't reproduce the issue but it looks like a race condition. Adding an artificial pause in the flycast code did trigger it. I pushed a workaround (0a526d7c94f53087a90d084003f92007e3991865) and fixed a few other things.
Unfortunately it seems shifting the problem downstream. Now it does not crash (of course, due to the removal of the verify), but it gets stuck on a mutex wait there: https://github.com/flyinghead/flycast/blob/28ea3dc69c1859ac77ad3fa6fc16410b1b8f01e6/core/emulator.cpp#L1018 Called from: https://github.com/flyinghead/flycast/blob/28ea3dc69c1859ac77ad3fa6fc16410b1b8f01e6/core/emulator.cpp#L805 Itself called from: https://github.com/flyinghead/flycast/blob/28ea3dc69c1859ac77ad3fa6fc16410b1b8f01e6/core/emulator.cpp#L869
For most of the steps it was okay, but on the opposite, for some others, it never stops, I have to manually pause, and then I can step again.
After retrying multiple times, it seems mostly OK, I did only fall in the trap (pun intended) while stepping at the beginning of the program. But there was a lot of inconsistencies, like the program not stopping at some instructions, or the breakpoints not being added if I do not pause, then continue manually after the startup.
The missing breakpoints at startup are due to the elf binary being loaded concurrently to the breakpoints being written to memory, some of which may be overwritten. Not ideal... but there's no easy fix. I'll try to find some time to do more serious work in this area.