esp-exception-decoder icon indicating copy to clipboard operation
esp-exception-decoder copied to clipboard
verified publisher icon dankeboy36

Exception decoding for RISC-V chips (ESP32-C3, C6, H2, etc)

Open igrr opened this issue 2 years ago • 4 comments

Hi @dankeboy36, thanks a lot for effort you put into making this incredibly useful plugin! I would like to check if you have already looked into implementing exception decoding for RISC-V chips, such as ESP32-C3? I found one mention of C3 in the tests but it seems that the code required to convert ESP32-C3 register+stack dumps into backtraces is missing.

Do you have something like that in progress, do you plan on implementing this, or would you like anyone from Espressif to open a PR with this implementation?

For reference: on RISC-V chips, by default we can only dump the registers and the stack, not the backtrace. Unwinding the stack to produce the backtrace on the chip itself is not possible, unless CONFIG_ESP_SYSTEM_USE_EH_FRAME option is enabled. (It is disabled by default, since it significantly increases application binary size.)

To perform stack unwinding, a tool running on the host PC is required. In ESP-IDF, this is done using gdb_panic_server.py which acts as a GDB server. GDB is launched, it connects to the script acting as a server, the script feeds GDB information from the stack dump as if it was coming from the real target, and finally GDB decodes the backtrace (code here).

This is slightly more complex than decoding backtraces on Xtensa chips, but since we already have access to GDB in this plugin, looks like this should be doable.

igrr avatar Jul 13 '23 15:07 igrr

Thank you for the feature request.

if you have already looked into implementing exception decoding for RISC-V chips, such as ESP32-C3?

No, I have not.

I found one mention of C3 in the tests

Great that you have looked into the code. :pray:

but it seems that the code required to convert ESP32-C3 register+stack dumps into backtraces is missing.

The tests cover the correctness of the gdb path lookup based on the FQBN. The decoding logic needs to be implemented.

Do you have something like that in progress,

No, I do not have anything.

do you plan on implementing this,

Of course, why not? Great idea.

or would you like anyone from Espressif to open a PR with this implementation?

I always appreciate any help you can give me.

I am neither a domain expert nor programming Arduino boards; I enjoy writing tooling and understand the Arduino ecosystem well, so any technical pointer, as you gave above, is welcome.

If you plan to contribute, developing a working prototype is sufficient, and I can help finalize the code. Please let me know if you decide so.

Otherwise, I am also happy to understand how it works and reimplement it, but it will be available later as I have other priorities. Also, I need to order a C3 board and look into the Python code. So please bear with me if you decide to wait.

I would be happy to have this new feature in the repository.

dankeboy36 avatar Jul 15 '23 10:07 dankeboy36

I would be very interested. I just realised I don't have a backtrace as I'm using an ESP32C3. I'm trying to find a way to see where my code causes a random Load access fault.

majodi avatar Sep 21 '23 13:09 majodi

this is done using gdb_panic_server.py

It's a 404 now. Could you please share a permalink? Thank you!

dankeboy36 avatar Oct 12 '23 19:10 dankeboy36

Here it is @dankeboy36:

https://github.com/espressif/esp-idf-monitor/blob/fae383ecf281655abaa5e65433f671e274316d10/esp_idf_monitor/gdb_panic_server.py

The removal/replacement occurred here: https://github.com/espressif/esp-idf-monitor/commit/dc672d682799e642aee2554f7733d38d4bbdba75

per1234 avatar Oct 12 '23 20:10 per1234

From https://github.com/dankeboy36/esp-exception-decoder/issues/31#issuecomment-2499792061:

I have a similar problem but this has nothing to do with the copy/paste. The decoder reports "Could not recognize stack trace/backtrace" after pasting the entire dump from the serial stack trace.

Board is ESP32-C3 (XIAO_ESP32C3). I also tried the Ardu Uno Rev 4 (that's also ESP32 based) but that comes back with a "Unsupported board: 'arduino:renesas_uno:unor4wifi'" error. 2 Conclusions: 1) the C3 seems supported (but does not work); 2) the Uno R4 should also be added since that would be the same core.

For pasting the stack I tried a number of things, paste only the stack trace without the registry dump, paste without the top error line, paste without the ELF SHA line and more, none of those worked.

For repro here's the code:

void setup() {
  // put your setup code here, to run once:
  Serial.begin(115200);
  Serial.println("SETUP");
}

class a
{
public:
  int a = 323;
  int geta() {return a;}
};

void loop() {
  // put your main code here, to run repeatedly:
  Serial.println("LOOP");
  delay(2000);
  a* _a = 0;
  //a* _a = new a();
  Serial.println(_a->geta());
}

stack trace:

Guru Meditation Error: Core  0 panic'ed (Load access fault). Exception was unhandled.

Core  0 register dump:
MEPC    : 0x42000074  RA      : 0x42000072  SP      : 0x3fc94f70  GP      : 0x3fc8c000  
TP      : 0x3fc8830c  T0      : 0x4005890e  T1      : 0x18000000  T2      : 0x00000000  
S0/FP   : 0x3fc8d000  S1      : 0x00000000  A0      : 0x00000001  A1      : 0x00000001  
A2      : 0x0000000a  A3      : 0x00000004  A4      : 0x600c0000  A5      : 0x00000000  
A6      : 0xfa000000  A7      : 0x00000003  S2      : 0x00000000  S3      : 0x00000000  
S4      : 0x00000000  S5      : 0x00000000  S6      : 0x00000000  S7      : 0x00000000  
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000  
T3      : 0x3fc95480  T4      : 0x00000000  T5      : 0x00000000  T6      : 0x00000000  
MSTATUS : 0x00001881  MTVEC   : 0x40380001  MCAUSE  : 0x00000005  MTVAL   : 0x00000000  
MHARTID : 0x00000000  

Stack memory:
3fc94f70: 0x00000000 0x00000000 0x00000000 0x4200360a 0x00000000 0x00000000 0x00000000 0x403872d8
3fc94f90: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc94fb0: 0x00000000 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xbaad5678 0x00000160 0xabba1234
3fc94fd0: 0x00000154 0x3fc94ed0 0x000007d6 0x3fc8d0e8 0x3fc8d0e8 0x3fc94fd4 0x3fc8d0e0 0x00000018
3fc94ff0: 0x5c7915e7 0x7b4e7a8c 0x3fc94fd4 0x00000000 0x00000001 0x3fc92fc4 0x706f6f6c 0x6b736154
3fc95010: 0xcfb6bc00 0x0093573e 0x00000000 0x3fc94fc0 0x00000001 0x00000000 0x00000000 0x00000000
3fc95030: 0x00000000 0x3fc8ed50 0x3fc8edb8 0x3fc8ee20 0x00000000 0x00000000 0x00000001 0x00000000
3fc95050: 0x00000000 0x00000000 0x4201a80c 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc95070: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc95090: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc950b0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc950d0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc950f0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc95110: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x02000000 0xbaad5678 0x00000060
3fc95130: 0xabba1234 0x00000054 0x00000000 0x3fc95138 0x00000000 0x00000000 0x00000000 0x3fc95150
3fc95150: 0xffffffff 0x3fc95150 0x3fc95150 0x00000000 0x3fc95164 0xffffffff 0x3fc95164 0x3fc95164
3fc95170: 0x00000001 0x00000001 0x00000000 0x0100ffff 0x00000000 0xb33fffff 0x00000000 0xbaad5678
3fc95190: 0x00000160 0xabba1234 0x00000154 0x3fc951f0 0x3fc951f0 0x3fc952f0 0x3fc952ef 0x00000000
3fc951b0: 0x3fc951b4 0xffffffff 0x3fc951b4 0x3fc951b4 0x00000000 0x3fc951c8 0xffffffff 0x3fc951c8
3fc951d0: 0x3fc951c8 0x00000000 0x00000100 0x00000001 0xdf00ffff 0x00000000 0xb33fffff 0x00000000
3fc951f0: 0x1b5a20c3 0xee7e423f 0x658fe1be 0x6dbb0365 0x2768218d 0x1f90cd68 0x182c00b2 0x4294b268
3fc95210: 0xa04df2fd 0x75048135 0x03eb0af3 0xeb7eda73 0x24338fa1 0xf7ed1cd7 0x8e5fc680 0x1979b0ae
3fc95230: 0xd3ef46f1 0x6372363e 0x38a11040 0x2e8efe8a 0x6d068dd2 0x8b00a172 0xb9830d9c 0x8769eab7
3fc95250: 0xa7c8daf7 0xbbb33bb5 0xbaf6b4e8 0xbf37a935 0x486d4432 0x3d990202 0x170898bb 0x97abc2b6
3fc95270: 0x4eb7fc73 0x055c4723 0x0733dec4 0xb5069893 0x66965406 0x8d46a1ec 0x36c1315a 0xdae43046
3fc95290: 0x20715846 0x6f1f2f25 0x9bd65078 0x91c8d748 0x3fdf2491 0xd4861e1d 0x9098b0f1 0x10529681
3fc952b0: 0xe841529d 0x1612cef2 0xa98be51a 0x15632d7b 0x46697688 0xfd75400a 0xce0eee80 0xdf004ebd
3fc952d0: 0x0de9176e 0xb8bb67ef 0x0502664c 0x5f7bceb7 0x06a6cfd2 0x30580f0e 0x6921bcb0 0xad210049
3fc952f0: 0xbaad5678 0x0000007c 0xabba1234 0x00000070 0x00000100 0x00000100 0x00000002 0x40383d14
3fc95310: 0x40384394 0x40383c4a 0x403848dc 0x403848c0 0x3fc9538d 0x3fc9538d 0x3fc95380 0x3fc95380
3fc95330: 0x3fc95380 0x3fc95480 0x0000000d 0x00000000 0x3fc95344 0xffffffff 0x3fc95344 0x3fc95344
3fc95350: 0x00000000 0x3fc95358 0xffffffff 0x3fc95358 0x3fc95358 0x00000000 0xb33fffff 0x00000000

ELF file SHA256: d0e2e7e7f0a0afca

Exception decoder panel dump:

ESP Exception Decoder
Sketch: sketch_nov25a FQBN: esp32:esp32:XIAO_ESP32C3

Guru Meditation Error: Core  0 panic'ed (Load access fault). Exception was unhandled.

Core  0 register dump:
MEPC    : 0x42000074  RA      : 0x42000072  SP      : 0x3fc94f70  GP      : 0x3fc8c000  
TP      : 0x3fc8830c  T0      : 0x4005890e  T1      : 0x18000000  T2      : 0x00000000  
S0/FP   : 0x3fc8d000  S1      : 0x00000000  A0      : 0x00000001  A1      : 0x00000001  
A2      : 0x0000000a  A3      : 0x00000004  A4      : 0x600c0000  A5      : 0x00000000  
A6      : 0xfa000000  A7      : 0x00000003  S2      : 0x00000000  S3      : 0x00000000  
S4      : 0x00000000  S5      : 0x00000000  S6      : 0x00000000  S7      : 0x00000000  
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000  
T3      : 0x3fc95480  T4      : 0x00000000  T5      : 0x00000000  T6      : 0x00000000  
MSTATUS : 0x00001881  MTVEC   : 0x40380001  MCAUSE  : 0x00000005  MTVAL   : 0x00000000  
MHARTID : 0x00000000  

Stack memory:
3fc94f70: 0x00000000 0x00000000 0x00000000 0x4200360a 0x00000000 0x00000000 0x00000000 0x403872d8
3fc94f90: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc94fb0: 0x00000000 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xbaad5678 0x00000160 0xabba1234
3fc94fd0: 0x00000154 0x3fc94ed0 0x000007d6 0x3fc8d0e8 0x3fc8d0e8 0x3fc94fd4 0x3fc8d0e0 0x00000018
3fc94ff0: 0x5c7915e7 0x7b4e7a8c 0x3fc94fd4 0x00000000 0x00000001 0x3fc92fc4 0x706f6f6c 0x6b736154
3fc95010: 0xcfb6bc00 0x0093573e 0x00000000 0x3fc94fc0 0x00000001 0x00000000 0x00000000 0x00000000
3fc95030: 0x00000000 0x3fc8ed50 0x3fc8edb8 0x3fc8ee20 0x00000000 0x00000000 0x00000001 0x00000000
3fc95050: 0x00000000 0x00000000 0x4201a80c 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc95070: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc95090: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc950b0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc950d0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc950f0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
3fc95110: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x02000000 0xbaad5678 0x00000060
3fc95130: 0xabba1234 0x00000054 0x00000000 0x3fc95138 0x00000000 0x00000000 0x00000000 0x3fc95150
3fc95150: 0xffffffff 0x3fc95150 0x3fc95150 0x00000000 0x3fc95164 0xffffffff 0x3fc95164 0x3fc95164
3fc95170: 0x00000001 0x00000001 0x00000000 0x0100ffff 0x00000000 0xb33fffff 0x00000000 0xbaad5678
3fc95190: 0x00000160 0xabba1234 0x00000154 0x3fc951f0 0x3fc951f0 0x3fc952f0 0x3fc952ef 0x00000000
3fc951b0: 0x3fc951b4 0xffffffff 0x3fc951b4 0x3fc951b4 0x00000000 0x3fc951c8 0xffffffff 0x3fc951c8
3fc951d0: 0x3fc951c8 0x00000000 0x00000100 0x00000001 0xdf00ffff 0x00000000 0xb33fffff 0x00000000
3fc951f0: 0x1b5a20c3 0xee7e423f 0x658fe1be 0x6dbb0365 0x2768218d 0x1f90cd68 0x182c00b2 0x4294b268
3fc95210: 0xa04df2fd 0x75048135 0x03eb0af3 0xeb7eda73 0x24338fa1 0xf7ed1cd7 0x8e5fc680 0x1979b0ae
3fc95230: 0xd3ef46f1 0x6372363e 0x38a11040 0x2e8efe8a 0x6d068dd2 0x8b00a172 0xb9830d9c 0x8769eab7
3fc95250: 0xa7c8daf7 0xbbb33bb5 0xbaf6b4e8 0xbf37a935 0x486d4432 0x3d990202 0x170898bb 0x97abc2b6
3fc95270: 0x4eb7fc73 0x055c4723 0x0733dec4 0xb5069893 0x66965406 0x8d46a1ec 0x36c1315a 0xdae43046
3fc95290: 0x20715846 0x6f1f2f25 0x9bd65078 0x91c8d748 0x3fdf2491 0xd4861e1d 0x9098b0f1 0x10529681
3fc952b0: 0xe841529d 0x1612cef2 0xa98be51a 0x15632d7b 0x46697688 0xfd75400a 0xce0eee80 0xdf004ebd
3fc952d0: 0x0de9176e 0xb8bb67ef 0x0502664c 0x5f7bceb7 0x06a6cfd2 0x30580f0e 0x6921bcb0 0xad210049
3fc952f0: 0xbaad5678 0x0000007c 0xabba1234 0x00000070 0x00000100 0x00000100 0x00000002 0x40383d14
3fc95310: 0x40384394 0x40383c4a 0x403848dc 0x403848c0 0x3fc9538d 0x3fc9538d 0x3fc95380 0x3fc95380
3fc95330: 0x3fc95380 0x3fc95480 0x0000000d 0x00000000 0x3fc95344 0xffffffff 0x3fc95344 0x3fc95344
3fc95350: 0x00000000 0x3fc95358 0xffffffff 0x3fc95358 0x3fc95358 0x00000000 0xb33fffff 0x00000000

ELF file SHA256: d0e2e7e7f0a0afca

Could not recognize stack trace/backtrace

Paste exception to decode...

dankeboy36 avatar Nov 26 '24 07:11 dankeboy36

Any update on support for these ESP's? I'm having an ESP32-C6 here for which I was trying the decoder but also ending up in Could not recognize stack trace/backtrace

dirkvranckaert avatar Dec 02 '24 10:12 dirkvranckaert

+1

ElFishi avatar Feb 17 '25 12:02 ElFishi

Please help test https://github.com/dankeboy36/esp-exception-decoder/pull/37. Thank you!

dankeboy36 avatar Mar 01 '25 21:03 dankeboy36

Testing as we speak. So far so good 👍 Thanks a lot! Any idea when this will get released in the main version?

dirkvranckaert avatar Mar 03 '25 05:03 dirkvranckaert

Also: Can we use this with a stacktrace for a custom ELF file? In order to investigate production issues logged on previous releases?

dirkvranckaert avatar Mar 03 '25 06:03 dirkvranckaert

:tada: This issue has been resolved in version 1.1.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

dankeboy36 avatar Mar 06 '25 19:03 dankeboy36