esp-idf icon indicating copy to clipboard operation
esp-idf copied to clipboard

ESP32C6: regdma_link_get_next hit Guru Meditation Error: Core 0 panic'ed (Stack protection fault) in master (IDFGH-12238)

Open AxelLin opened this issue 1 year ago • 18 comments

Answers checklist.

  • [X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
  • [X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
  • [X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.3-dev-2218-g7cf8cc79a1

Espressif SoC revision.

ESP32C6

Operating System used.

Linux

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

Development Kit.

Custom Board

Power Supply used.

USB

What is the expected behavior?

no panic, as exactly the same application code works in v5.2 branch.

What is the actual behavior?

Guru Meditation Error: Core 0 panic'ed (Stack protection fault).

Steps to reproduce.

My application is running nimble host-only using uart hci but I cannot share my application code.

Debug Logs.

Guru Meditation Error: Core  0 panic'ed (Stack protection fault).

Detected in task "nimble_host" at 0x4200471c
0x4200471c: regdma_link_get_next at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:351

Stack pointer: 0x4084de90
Stack bounds: 0x4084dea8 - 0x4084eea0


Stack dump detected
Core  0 register dump:
MEPC    : 0x42004750  RA      : 0x420047cc  SP      : 0x4084de90  GP      : 0x40818354
0x42004750: regdma_link_get_next at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:363
0x420047cc: regdma_link_recursive_impl at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390 (discriminator 1)
 (inlined by) regdma_link_recursive_impl at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385 (discriminator 1)

TP      : 0x40818a7c  T0      : 0x40022494  T1      : 0x4080288a  T2      : 0xffffffff
0x40022494: multi_heap_internal_unlock in ROM
0x4080288a: uart_ll_disable_intr_mask at /home/axel/esp/esp-idf/components/hal/esp32c6/include/hal/uart_ll.h:445
 (inlined by) uart_rx_intr_handler_default at /home/axel/esp/esp-idf/components/esp_driver_uart/src/uart.c:914

S0/FP   : 0x40828620  S1      : 0x00000003  A0      : 0x40828620  A1      : 0x00000000
A2      : 0x00000052  A3      : 0x420048d0  A4      : 0x70030000  A5      : 0x4210b000
0x420048d0: regdma_link_update_stats_wrapper at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:442

A6      : 0x00000004  A7      : 0x0000000a  S2      : 0x00000000  S3      : 0x00000052
S4      : 0x00000004  S5      : 0x00000006  S6      : 0x00000000  S7      : 0x00000000
S8      : 0x00000003  S9      : 0x00000004  S10     : 0x42c80000  S11     : 0x41200000
T3      : 0x00000000  T4      : 0x00000001  T5      : 0x851eb850  T6      : 0x00000001
MSTATUS : 0x00001881  MTVEC   : 0x40800001  MCAUSE  : 0x0000001b  MTVAL   : 0xc8078793
0x40800001: _vector_table at ??:?

MHARTID : 0x00000000


Backtrace:


0x42004750 in regdma_link_get_next (link=link@entry=0x40828620, entry=entry@entry=0) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:363
363                 regdma_entry_buf_t *next = (*nextfn1[head.mode])(link);
#0  0x42004750 in regdma_link_get_next (link=link@entry=0x40828620, entry=entry@entry=0) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:363
#1  0x420047cc in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=82, entry=0, link=0x40828620) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#2  regdma_link_recursive_impl (link=0x40828620, entry=0, depth=82, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#3  0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=81, entry=0, link=0x4082864c) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#4  regdma_link_recursive_impl (link=0x4082864c, entry=0, depth=81, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#5  0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=80, entry=0, link=0x40828678) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#6  regdma_link_recursive_impl (link=0x40828678, entry=0, depth=80, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#7  0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=79, entry=0, link=0x40828714) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#8  regdma_link_recursive_impl (link=0x40828714, entry=0, depth=79, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#9  0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=78, entry=0, link=0x40828738) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#10 regdma_link_recursive_impl (link=0x40828738, entry=0, depth=78, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#11 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=77, entry=0, link=0x408287fc) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#12 regdma_link_recursive_impl (link=0x408287fc, entry=0, depth=77, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#13 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=76, entry=0, link=0x408288f8) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#14 regdma_link_recursive_impl (link=0x408288f8, entry=0, depth=76, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#15 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=75, entry=0, link=0x40828994) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#16 regdma_link_recursive_impl (link=0x40828994, entry=0, depth=75, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#17 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=74, entry=0, link=0x408289b8) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#18 regdma_link_recursive_impl (link=0x408289b8, entry=0, depth=74, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#19 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=73, entry=0, link=0x40828a7c) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#20 regdma_link_recursive_impl (link=0x40828a7c, entry=0, depth=73, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#21 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=72, entry=0, link=0x40828b78) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#22 regdma_link_recursive_impl (link=0x40828b78, entry=0, depth=72, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#23 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=71, entry=0, link=0x40828ba4) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#24 regdma_link_recursive_impl (link=0x40828ba4, entry=0, depth=71, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#25 0x420047d6 in regdma_link_recursive_impl (hook=0x420048d0 <regdma_link_update_stats_wrapper>, depth=70, entry=0, link=0x40828bd0) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#26 regdma_link_recursive_impl (link=0x40828bd0, entry=0, depth=70, hook=0x420048d0 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#27 0x00000000 in ?? ()
Backtrace stopped: frame did not save the PC
ELF file SHA256: ed9c36e055e3fa76

Rebooting...
ESP-ROM:esp32c6-20220919

More Information.

No response

AxelLin avatar Feb 29 '24 02:02 AxelLin

ff94741cd13fbd43a3805ead2b60cc59ebb8f212 is good commit, 2302dd5a91cac6dce2853feda780a1b90a056001 is bad commit.

So it looks like below commit causes this regression: Merge branch 'feature/i2c_sleep' into 'master'

feat(i2c): Support i2c sleep retention on esp32c6/h2

AxelLin avatar Feb 29 '24 06:02 AxelLin

@AxelLin Please try to increase the size of CONFIG_BT_NIMBLE_HOST_TASK_STACK_SIZE in menuconfig.

esp-wzh avatar Feb 29 '24 06:02 esp-wzh

@AxelLin Please try to increase the size of CONFIG_BT_NIMBLE_HOST_TASK_STACK_SIZE in menuconfig.

Why? The error is not stack overflow. Anyway , tested CONFIG_BT_NIMBLE_HOST_TASK_STACK_SIZE=8192 still hit the same error.

AxelLin avatar Feb 29 '24 06:02 AxelLin

Guru Meditation Error: Core 0 panic'ed (Stack protection fault).

It is indeed caused by stack overflow, refer: https://docs.espressif.com/projects/esp-idf/en/v5.2/esp32c6/api-guides/fatal-errors.html#stack-overflow

I'm guessing it's caused by the fact that with i2c sleep retention support, the depth of function calls in the sleep retention de-initialize flow has become deeper.

esp-wzh avatar Feb 29 '24 07:02 esp-wzh

@esp-wzh It's in reviewing state, I'm wondering if you can provide the patch for testing.

AxelLin avatar Mar 04 '24 07:03 AxelLin

@AxelLin

Stack pointer: 0x4084de90
Stack bounds: 0x4084dea8 - 0x4084eea0

I'm not sure how to reproduce the problem you're having, from the log attached, task nimble_host has a stack region of 0x4084dea8 - 0x4084eea0, but during the task run the Stack Pointer crosses over the minimum address.

Can you share the log when the stack overflowed after you increased the size of CONFIG_BT_NIMBLE_HOST_TASK_STACK_SIZE to 8192 Byte ?

Or can you help provide a minimum reproduction example?

esp-wzh avatar Mar 04 '24 08:03 esp-wzh

I'm guessing it's caused by the fact that with i2c sleep retention support, the depth of function calls in the sleep retention de-initialize flow has become deeper.

@esp-wzh

I tested again and now I notice the stackover flow could happen on various tasks. This is a problem, which means I need to increase stacksize for various tasks.

Below is an example of the issue happen on "main" task.


Detected in task "main" at 0x420049e2
0x420049e2: regdma_link_get_next at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:351

Stack pointer: 0x4082b080
Stack bounds: 0x4082b090 - 0x4082c280


Stack dump detected
Core  0 register dump:
MEPC    : 0x42004a84  RA      : 0x42004ade  SP      : 0x4082b080  GP      : 0x40819b44  
0x42004a84: regdma_link_get_next at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:376
0x42004ade: regdma_link_recursive_impl at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390 (discriminator 1)
 (inlined by) regdma_link_recursive_impl at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385 (discriminator 1)

TP      : 0x4082c280  T0      : 0x40022494  T1      : 0x40802aa0  T2      : 0xffffffff  
0x40022494: multi_heap_internal_unlock in ROM
0x40802aa0: uart_ll_disable_intr_mask at /home/axel/esp/esp-idf/components/hal/esp32c6/include/hal/uart_ll.h:445
 (inlined by) uart_rx_intr_handler_default at /home/axel/esp/esp-idf/components/esp_driver_uart/src/uart.c:914

S0/FP   : 0x40835fa8  S1      : 0x42004c12  A0      : 0x40835fa8  A1      : 0x00000000  
0x42004c12: regdma_link_update_stats_wrapper at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:442

A2      : 0x00000067  A3      : 0x42004c12  A4      : 0x00000003  A5      : 0x00000003  
0x42004c12: regdma_link_update_stats_wrapper at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:442

A6      : 0x00000004  A7      : 0x0000000a  S2      : 0x00000000  S3      : 0x00000067  
S4      : 0x00000004  S5      : 0x00000006  S6      : 0x00000000  S7      : 0x00000000  
S8      : 0x40825000  S9      : 0x40825000  S10     : 0x00000000  S11     : 0x00000000  
T3      : 0x00000000  T4      : 0x00000001  T5      : 0x40825000  T6      : 0x00000001  
MSTATUS : 0x00001881  MTVEC   : 0x40800001  MCAUSE  : 0x0000001b  MTVAL   : 0x00004732  
0x40800001: _vector_table at ??:?

MHARTID : 0x00000000  


Backtrace:


regdma_link_get_next (link=link@entry=0x40835fa8, entry=entry@entry=0) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:376
376	            void *next = (*nextfn0[head.mode])(link);
#0  regdma_link_get_next (link=link@entry=0x40835fa8, entry=entry@entry=0) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:376
#1  0x42004ade in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=103, entry=0, link=0x40835fa8) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#2  regdma_link_recursive_impl (link=0x40835fa8, entry=0, depth=103, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#3  0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=102, entry=0, link=0x408360ac) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#4  regdma_link_recursive_impl (link=0x408360ac, entry=0, depth=102, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#5  0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=101, entry=0, link=0x408360cc) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#6  regdma_link_recursive_impl (link=0x408360cc, entry=0, depth=101, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#7  0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=100, entry=0, link=0x408360ec) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#8  regdma_link_recursive_impl (link=0x408360ec, entry=0, depth=100, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#9  0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=99, entry=0, link=0x4083610c) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#10 regdma_link_recursive_impl (link=0x4083610c, entry=0, depth=99, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#11 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=98, entry=0, link=0x40836184) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#12 regdma_link_recursive_impl (link=0x40836184, entry=0, depth=98, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#13 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=97, entry=0, link=0x40829fac) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#14 regdma_link_recursive_impl (link=0x40829fac, entry=0, depth=97, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#15 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=96, entry=0, link=0x40829fd8) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#16 regdma_link_recursive_impl (link=0x40829fd8, entry=0, depth=96, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#17 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=95, entry=0, link=0x40829ff8) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#18 regdma_link_recursive_impl (link=0x40829ff8, entry=0, depth=95, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#19 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=94, entry=0, link=0x4082a020) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#20 regdma_link_recursive_impl (link=0x4082a020, entry=0, depth=94, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#21 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=93, entry=0, link=0x4082a040) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#22 regdma_link_recursive_impl (link=0x4082a040, entry=0, depth=93, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#23 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=92, entry=0, link=0x4082a060) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#24 regdma_link_recursive_impl (link=0x4082a060, entry=0, depth=92, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#25 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=91, entry=0, link=0x4082a080) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#26 regdma_link_recursive_impl (link=0x4082a080, entry=0, depth=91, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#27 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=90, entry=0, link=0x4082a0a0) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#28 regdma_link_recursive_impl (link=0x4082a0a0, entry=0, depth=90, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#29 0x42004ae8 in regdma_link_recursive_impl (hook=0x42004c12 <regdma_link_update_stats_wrapper>, depth=89, entry=0, link=0x4082a1c0) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:390
#30 regdma_link_recursive_impl (link=0x4082a1c0, entry=0, depth=89, hook=0x42004c12 <regdma_link_update_stats_wrapper>) at /home/axel/esp/esp-idf/components/esp_hw_support/port/regdma_link.c:385
#31 0x00000000 in ?? ()
Backtrace stopped: frame did not save the PC
ELF file SHA256: 11864d7830376559

Rebooting...
ESP-ROM:esp32c6-20220919
Build:Sep 19 2022
rst:0xc (SW_CPU),boot:0x6c (SPI_FAST_FLASH_BOOT)



AxelLin avatar Mar 04 '24 08:03 AxelLin

@esp-wzh

FYI, see the HWM changes:

v5.1 branch: Task Name Status Prio HWM Task# nimble_host B 21 5592 10

master: Task Name Status Prio HWM Task# nimble_host B 21 3176 10

master (ff94741, before supporting i2c sleep retention on esp32c6/h2) Task Name Status Prio HWM Task# nimble_host B 21 5608 10

AxelLin avatar Mar 04 '24 09:03 AxelLin

Can you briefly describe what you did in the main task and how it called into function regdma_link?

Usually regdma_link_recursive_impl will be called when module deinit destroys its retention link. as you can see in the backtrace, we currently use a recursive algorithm to search for nodes that need to be destroyed in the linked list.

I have encountered this problem before, and realized that this algorithm is not suitable for application in embedded systems with scarce RAM. We are exploring better solutions, which are already in our TODO List.

As analyzed above, if the module de-initialization(bt, i2c, gdma ...) is called by a fixed task, you can temporarily avoid this problem by increasing the stack size of the corresponding task.

esp-wzh avatar Mar 04 '24 09:03 esp-wzh

@esp-wzh

FYI, see the HWM changes:

v5.1 branch: Task Name Status Prio HWM Task# nimble_host B 21 5592 10

master: Task Name Status Prio HWM Task# nimble_host B 21 3176 10

master (ff94741, before supporting i2c sleep retention on esp32c6/h2) Task Name Status Prio HWM Task# nimble_host B 21 5608 10

It makes sense because after i2c retention is supported, destroying the link requires traversing more nodes, going through deeper recursion, and using more of the stack.

I made a simple attempt. Adding the FORCE_INLINE_ATTR attribute to the regdma_link_get_next can save about 25% of the call stack consumption. It may be helpful to you.

esp-wzh avatar Mar 04 '24 09:03 esp-wzh

@esp-wzh FYI, see the HWM changes: v5.1 branch: Task Name Status Prio HWM Task# nimble_host B 21 5592 10 master: Task Name Status Prio HWM Task# nimble_host B 21 3176 10 master (ff94741, before supporting i2c sleep retention on esp32c6/h2) Task Name Status Prio HWM Task# nimble_host B 21 5608 10

It makes sense because after i2c retention is supported, destroying the link requires traversing more nodes, going through deeper recursion, and using more of the stack.

It takes 2KB+ stack size in current master (compare to master (ff94741)). I don't think this is ok, which is likely to cause regression for existing application.

AxelLin avatar Mar 04 '24 11:03 AxelLin

@esp-wzh

Just remind this issue can happen on different tasks. Please fix this, otherwise the master tree is simply broken. It's impossible to increase 2KB+ stack size for each task.

AxelLin avatar Mar 13 '24 09:03 AxelLin

@esp-wzh

I'm still waiting for this fix, any update? If it really needs much longer time to fix it, I think you should revert the change.

AxelLin avatar Apr 09 '24 02:04 AxelLin

Can you share a minimal demo that reproduces this issue? @AxelLin

esp-wzh avatar Apr 09 '24 04:04 esp-wzh

Can you share a minimal demo that reproduces this issue? @AxelLin

Sigh, which means no progress in the past month. too bad. You are aware the issue is related to stack size, which means you can check task HWM to understand why it causes issue. (https://github.com/espressif/esp-idf/issues/13288#issuecomment-1976081673) I believe you can easily reproduce the issue with adjusting the stack size.

AxelLin avatar Apr 09 '24 04:04 AxelLin

@AxelLin I added the method of using iterators to optimize the stack consumption caused by recursion. The following figure shows the comparison before and after optimization (refer to the attachments regdma link recursive.log and regdma link iterator.log for detailed log files), and the patch file of optimization implementation refer to 0001-fix-esp_hw_support-fixed-stack-protection-fault-caus.patch. 2024-04-10_174731_recursive 2024-04-10_172217_iterator

esp-lis avatar Apr 10 '24 10:04 esp-lis

@esp-lis @esp-wzh

Yes, the patch fixes the panic. Above log already shows the big difference about stack consumption: 3.2KB v.s. 48 bytes.

Please fix it ASAP, since this is a regression. BTW, the v5.2 branch (v5.2.1-443-g2ba5320112) also needs fix now.

AxelLin avatar Apr 24 '24 01:04 AxelLin

@esp-lis @esp-wzh As I said this regression is now in v5.2 release branch, so when will this get fixed?

AxelLin avatar May 05 '24 03:05 AxelLin

@esp-lis @esp-wzh As I said this regression is now in v5.2 release branch, so when will this get fixed?

The fix is not yet available in v5.2. When will you backport the fix. (I have been wait for more than 1 Month)

AxelLin avatar Jun 17 '24 06:06 AxelLin