WIFI component: Allocate static buffer in SPIRAM, the other parts (dynamically buffer, ...) in internal RAM (IDFGH-10507)

[Slider0007]

Is your feature request related to a problem?

Running out of internal memory. WIFI component: With option CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP only all buffers, also dynamically allocated ones, are preferably stored into SPIRAM. This leads to fragmentation of SPIRAM.

Describe the solution you'd like.

It would be beneficial to have an additional option to allocate static buffers in SPIRAM and dynamically allocated buffers and other smaller components in internal RAM. This would free more internal RAM and avoid fragmentation in SPIRAM.

Describe alternatives you've considered.

N/A

Additional context.

N/A

[mhdong]

Hi @Slider0007 An additional configuration item, CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL, can be used to set the size threshold when a single allocation should prefer external memory. Please reference, https://github.com/espressif/esp-idf/blob/9f4e8eb0cde35308f2d46c2f419f4c170116bf9c/docs/en/api-guides/external-ram.rst?plain=1#L101

[Slider0007]

Hi @mhdong,

Thank you for your response.

I was playing with this parameter, but for me it seems that this is not working like expected. Even CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL is set to 8192, smaller chunks get allocated in SPIRAM.

---

Test 1: CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y

CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=8192
CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y

Dump from SPIRAM after bootup --> As you can see there are a lot of smaller chunks allocated in SPIRAM. When I switch off CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP, all those allocations are gone again --> Test 2

Showing data for heap: 0x3f802968 
Showing data for heap: 0x3f802968 
Block 0x3f8031e4 data, size: 55880 bytes, Free: No -> OK
Block 0x3f810c30 data, size: 12 bytes, Free: No
Block 0x3f810c40 data, size: 40 bytes, Free: No
Block 0x3f810c6c data, size: 16 bytes, Free: No
Block 0x3f810c80 data, size: 16 bytes, Free: No 
Block 0x3f810c94 data, size: 16 bytes, Free: No
Block 0x3f810ca8 data, size: 12 bytes, Free: No
Block 0x3f810cb8 data, size: 24 bytes, Free: No 
Block 0x3f810cd4 data, size: 2392 bytes, Free: No -> WIFI?
Block 0x3f811630 data, size: 136 bytes, Free: No
Block 0x3f8116bc data, size: 136 bytes, Free: No
Block 0x3f811748 data, size: 1696 bytes, Free: No -> From here static WIFI allocs?
Block 0x3f811dec data, size: 1696 bytes, Free: No
Block 0x3f812490 data, size: 1696 bytes, Free: No
Block 0x3f812b34 data, size: 1696 bytes, Free: No
Block 0x3f8131d8 data, size: 1696 bytes, Free: No 
Block 0x3f81387c data, size: 1696 bytes, Free: No
Block 0x3f813f20 data, size: 1696 bytes, Free: No
Block 0x3f8145c4 data, size: 1696 bytes, Free: No 
Block 0x3f814c68 data, size: 1696 bytes, Free: No
Block 0x3f81530c data, size: 1696 bytes, Free: No
Block 0x3f8159b0 data, size: 1696 bytes, Free: No
Block 0x3f816054 data, size: 1696 bytes, Free: No
Block 0x3f8166f8 data, size: 1696 bytes, Free: No
Block 0x3f816d9c data, size: 1696 bytes, Free: No 
Block 0x3f817440 data, size: 1696 bytes, Free: No
Block 0x3f817ae4 data, size: 1696 bytes, Free: No -> end
Block 0x3f818188 data, size: 572 bytes, Free: No -> WIFI?
Block 0x3f8183c8 data, size: 944 bytes, Free: No
Block 0x3f81877c data, size: 16 bytes, Free: No
Block 0x3f818790 data, size: 60 bytes, Free: No
Block 0x3f8187d0 data, size: 44 bytes, Free: No
Block 0x3f818800 data, size: 12 bytes, Free: No 
Block 0x3f818810 data, size: 184 bytes, Free: No
Block 0x3f8188cc data, size: 184 bytes, Free: No 
Block 0x3f818988 data, size: 40 bytes, Free: No
Block 0x3f8189b4 data, size: 16 bytes, Free: No
Block 0x3f8189c8 data, size: 12 bytes, Free: No
Block 0x3f8189d8 data, size: 20 bytes, Free: No
Block 0x3f8189f0 data, size: 64 bytes, Free: No
Block 0x3f818a34 data, size: 120 bytes, Free: No
Block 0x3f818ab0 data, size: 64 bytes, Free: No 
Block 0x3f818af4 data, size: 36 bytes, Free: No
Block 0x3f818b1c data, size: 40 bytes, Free: No
Block 0x3f818b48 data, size: 12 bytes, Free: No
Block 0x3f818b58 data, size: 168 bytes, Free: No 
Block 0x3f818c04 data, size: 48 bytes, Free: Yes
Block 0x3f818c38 data, size: 12 bytes, Free: No
Block 0x3f818c48 data, size: 168 bytes, Free: No 
Block 0x3f818cf4 data, size: 108 bytes, Free: Yes
Block 0x3f818d64 data, size: 52 bytes, Free: No
Block 0x3f818d9c data, size: 20 bytes, Free: No
Block 0x3f818db4 data, size: 56 bytes, Free: No
Block 0x3f818df0 data, size: 52 bytes, Free: No
Block 0x3f818e28 data, size: 12 bytes, Free: No
Block 0x3f818e38 data, size: 40 bytes, Free: No 
Block 0x3f818e64 data, size: 52 bytes, Free: No
Block 0x3f818e9c data, size: 20 bytes, Free: No
Block 0x3f818eb4 data, size: 136 bytes, Free: No -> end
Block 0x3f818f40 data, size: 61440 bytes, Free: No  -> OK, my application
Block 0x3f827f44 data, size: 168 bytes, Free: No
Block 0x3f827ff0 data, size: 16 bytes, Free: No
Block 0x3f828004 data, size: 16 bytes, Free: No
Block 0x3f828018 data, size: 20 bytes, Free: No 
Block 0x3f828030 data, size: 168 bytes, Free: No
Block 0x3f8280dc data, size: 12 bytes, Free: No
Block 0x3f8280ec data, size: 928 bytes, Free: No  -> OK, my application
Block 0x3f828490 data, size: 628 bytes, Free: No 
Block 0x3f828708 data, size: 28 bytes, Free: No
Block 0x3f828728 data, size: 16 bytes, Free: No
Block 0x3f82873c data, size: 16 bytes, Free: No
Block 0x3f828750 data, size: 168 bytes, Free: No 
Block 0x3f8287fc data, size: 308 bytes, Free: Yes
Block 0x3f828934 data, size: 168 bytes, Free: No
Block 0x3f8289e0 data, size: 136 bytes, Free: Yes
Block 0x3f828a6c data, size: 12 bytes, Free: No 
Block 0x3f828a7c data, size: 52 bytes, Free: No
Block 0x3f828ab4 data, size: 168 bytes, Free: No
Block 0x3f828b60 data, size: 52 bytes, Free: No
Block 0x3f828b98 data, size: 52 bytes, Free: No
Block 0x3f828bd0 data, size: 28 bytes, Free: Yes
Block 0x3f828bf0 data, size: 60 bytes, Free: No 
Block 0x3f828c30 data, size: 52 bytes, Free: No
Block 0x3f828c68 data, size: 76 bytes, Free: No
Block 0x3f828cb8 data, size: 28 bytes, Free: No
Block 0x3f828cd8 data, size: 1696 bytes, Free: No -> OK, my application
Block 0x3f82937c data, size: 1920 bytes, Free: No
Block 0x3f829b00 data, size: 10712 bytes, Free: No
Block 0x3f82c4dc data, size: 1920 bytes, Free: No -> end
Block 0x3f82cc60 data, size: 636 bytes, Free: Yes
Block 0x3f82cee0 data, size: 12 bytes, Free: No
Block 0x3f82cef0 data, size: 16 bytes, Free: No
Block 0x3f82cf04 data, size: 24576 bytes, Free: No -> OK, from here my application 
Block 0x3f832f08 data, size: 921604 bytes, Free: No 
Block 0x3f913f10 data, size: 128004 bytes, Free: No 
Block 0x3f933318 data, size: 226968 bytes, Free: No 
Block 0x3f96a9b4 data, size: 10712 bytes, Free: No 
Block 0x3f96d390 data, size: 1920 bytes, Free: No 
Block 0x3f96db14 data, size: 10712 bytes, Free: No 
Block 0x3f9704f0 data, size: 1920 bytes, Free: No 
Block 0x3f970c74 data, size: 10712 bytes, Free: No 
Block 0x3f973650 data, size: 1920 bytes, Free: No 
Block 0x3f973dd4 data, size: 10712 bytes, Free: No 
Block 0x3f9767b0 data, size: 47232 bytes, Free: No 
Block 0x3f982034 data, size: 3072 bytes, Free: No 
Block 0x3f982c38 data, size: 57132 bytes, Free: No 
Block 0x3f990b68 data, size: 3072 bytes, Free: No 
Block 0x3f99176c data, size: 57132 bytes, Free: No 
Block 0x3f99f69c data, size: 3072 bytes, Free: No 
Block 0x3f9a02a0 data, size: 57132 bytes, Free: No 
Block 0x3f9ae1d0 data, size: 3072 bytes, Free: No 
Block 0x3f9aedd4 data, size: 57132 bytes, Free: No -> end
Block 0x3f9bcd04 data, size: 1512 bytes, Free: No -> from here dynamic WIFI allocs?
Block 0x3f9bd2f0 data, size: 1512 bytes, Free: Yes
Block 0x3f9bd8dc data, size: 1512 bytes, Free: No
Block 0x3f9bdec8 data, size: 1512 bytes, Free: No
Block 0x3f9be4b4 data, size: 1512 bytes, Free: No
Block 0x3f9beaa0 data, size: 1512 bytes, Free: Yes -> end
Block 0x3f9bf08c data, size: 29312 bytes, Free: No 
Block 0x3f9c6310 data, size: 2333932 bytes, Free: Yes

---

Test 2: CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=n (all other parameter are untouched)

CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=8192
CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=n

Dump from SPIRAM after bootup

Showing data for heap: 0x3f802968 
Block 0x3f8031e4 data, size: 55880 bytes, Free: No -> OK
Block 0x3f810c30 data, size: 12 bytes, Free: No
Block 0x3f810c40 data, size: 56 bytes, Free: No
Block 0x3f810c7c data, size: 16 bytes, Free: Yes 
Block 0x3f810c90 data, size: 61440 bytes, Free: No  -> OK, from here my application
Block 0x3f81fc94 data, size: 928 bytes, Free: No
Block 0x3f820038 data, size: 1684 bytes, Free: No 
Block 0x3f8206d0 data, size: 1920 bytes, Free: No
Block 0x3f820e54 data, size: 1920 bytes, Free: No 
Block 0x3f8215d8 data, size: 1920 bytes, Free: No
Block 0x3f821d5c data, size: 1920 bytes, Free: No
Block 0x3f8224e0 data, size: 1920 bytes, Free: No 
Block 0x3f822c64 data, size: 48 bytes, Free: Yes
Block 0x3f822c98 data, size: 24576 bytes, Free: No
Block 0x3f828c9c data, size: 921604 bytes, Free: No 
Block 0x3f909ca4 data, size: 128004 bytes, Free: No
Block 0x3f9290ac data, size: 226968 bytes, Free: No
Block 0x3f960748 data, size: 10712 bytes, Free: No
Block 0x3f963124 data, size: 10712 bytes, Free: No 
Block 0x3f965b00 data, size: 10712 bytes, Free: No
Block 0x3f9684dc data, size: 10712 bytes, Free: No
Block 0x3f96aeb8 data, size: 10712 bytes, Free: No
Block 0x3f96d894 data, size: 47232 bytes, Free: No 
Block 0x3f979118 data, size: 3072 bytes, Free: No
Block 0x3f979d1c data, size: 57132 bytes, Free: No
Block 0x3f987c4c data, size: 3072 bytes, Free: No 
Block 0x3f988850 data, size: 57132 bytes, Free: No
Block 0x3f996780 data, size: 3072 bytes, Free: No
Block 0x3f997384 data, size: 57132 bytes, Free: No
Block 0x3f9a52b4 data, size: 3072 bytes, Free: No
Block 0x3f9a5eb8 data, size: 57132 bytes, Free: No -> end
Block 0x3f9b3de8 data, size: 2408980 bytes, Free: Yes 

Test 2 shows almost no fragmention, after activating CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP lots of smaller chunks from WIFI component gets allocated in SPIRAM. (Test 1).

It would be desirable if only the larger statically allocated blocks ended up in SPIRAM.

Thanks again for looking into this topic. If you need further info, don't hesitate to ask.

[mhdong]

Hi @Slider0007 If CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP is enabled. Prefer to allocate a chunk of memory in SPIRAM firstly. If failed, try to allocate it in internal memory then. CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP will cause CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL disable for WIFI and LWIP. We will add some relevant explanatory documents to explain this issue.

[Slider0007]

Hi @mhdong

Thanks for your explaination. I almost thought that this would be the case, but I do not understand why there is such an exception for WIFI and LWIP? Wouldn't it be possible to add a flag to choose which behaviour is better for user's use case (actual implemented alloc strategy or with respect of CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL like all other components)? Or if there is any limitation not be able to do so, implement a flag like I initially proposed -> allocate static chunks in SPIRAM and dynamically allocated chunks in internal memory.

Best Regards

[mhdong]

Hi @Slider0007

1. Yes, you can implement your own alloc strategy. /* If CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP is enabled. Prefer to allocate a chunk of memory in SPIRAM firstly. If failed, try to allocate it in internal memory then. */ IRAM_ATTR void *wifi_malloc( size_t size ) { #if CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP return heap_caps_malloc_prefer(size, 2, MALLOC_CAP_DEFAULT|MALLOC_CAP_SPIRAM, MALLOC_CAP_DEFAULT|MALLOC_CAP_INTERNAL); #else return malloc(size); #endif }

IRAM_ATTR void *wifi_malloc( size_t size ) { #if CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP if (size > SPIRAM_THRESHOLD) { return heap_caps_malloc_prefer(size, 2, MALLOC_CAP_DEFAULT|MALLOC_CAP_SPIRAM, MALLOC_CAP_DEFAULT|MALLOC_CAP_INTERNAL); } else { return malloc(size); } #else return malloc(size); #endif } 2. allocate static chunks in SPIRAM and dynamically allocated chunks in internal memory This strategy is not very easy to implement for customers. If there is indeed a need or significant performance improvement, we can also implement similar strategy in the future.

[Slider0007]

Hi @mhdong,

Thanks for your suggestion. Sounds at least like a possible workaround. How do replace this wifi malloc function the easiest way without touching the ESP-IDF installation? I tried to copy complete esp_wifi component to my project component folder but now it fails to compile due to dependencies of wifi component.

Thanks.

[mhdong]

Hi @Slider0007 You can find function wifi_malloc in file esp_adapter.c. Modifying this function in the function definition.

[Slider0007]

Hi @mhdong

I mananged to set this up (with a defined limit of 1600 byte) and tested for a longer period. I was able to get rid of the smaller dynamically allocated fragments related to WIFI component. Unfortunately this solve my problem only partially. There are still smaller fragments in SPIRAM even though I modified the malloc strategy of wifi component. Therefore SPIRAM is still fragmenting over time until system is running out of SPIRAM...

Showing data for heap: 0x3f802968 
Block 0x3f8031e4 data, size: 55880 bytes, Free: No
Block 0x3f810c30 data, size: 12 bytes, Free: No
Block 0x3f810c40 data, size: 40 bytes, Free: No
Block 0x3f810c6c data, size: 16 bytes, Free: No 
Block 0x3f810c80 data, size: 16 bytes, Free: No
Block 0x3f810c94 data, size: 16 bytes, Free: No
Block 0x3f810ca8 data, size: 12 bytes, Free: No
Block 0x3f810cb8 data, size: 2392 bytes, Free: No -> WIFI static allocations -> OK
Block 0x3f811614 data, size: 1696 bytes, Free: No 
Block 0x3f811cb8 data, size: 1696 bytes, Free: No
Block 0x3f81235c data, size: 1696 bytes, Free: No
Block 0x3f812a00 data, size: 1696 bytes, Free: No 
Block 0x3f8130a4 data, size: 1696 bytes, Free: No
Block 0x3f813748 data, size: 1696 bytes, Free: No
Block 0x3f813dec data, size: 1696 bytes, Free: No
Block 0x3f814490 data, size: 1696 bytes, Free: No
Block 0x3f814b34 data, size: 1696 bytes, Free: No
Block 0x3f8151d8 data, size: 1696 bytes, Free: No
Block 0x3f81587c data, size: 1696 bytes, Free: No
Block 0x3f815f20 data, size: 1696 bytes, Free: No 
Block 0x3f8165c4 data, size: 1696 bytes, Free: No
Block 0x3f816c68 data, size: 1696 bytes, Free: No
Block 0x3f81730c data, size: 1696 bytes, Free: No
Block 0x3f8179b0 data, size: 1696 bytes, Free: No -> WIFI static allocations -> OK
Block 0x3f818054 data, size: 16 bytes, Free: No -> ?
Block 0x3f818068 data, size: 16 bytes, Free: No
Block 0x3f81807c data, size: 16 bytes, Free: No
Block 0x3f818090 data, size: 56 bytes, Free: No
Block 0x3f8180cc data, size: 16 bytes, Free: No -> ?
Block 0x3f8180e0 data, size: 61440 bytes, Free: No -> My application -> OK
Block 0x3f8270e4 data, size: 52 bytes, Free: No -> Fro here ?
Block 0x3f82711c data, size: 12 bytes, Free: No 
Block 0x3f82712c data, size: 52 bytes, Free: No
Block 0x3f827164 data, size: 12 bytes, Free: No
Block 0x3f827174 data, size: 40 bytes, Free: No 
Block 0x3f8271a0 data, size: 52 bytes, Free: No
Block 0x3f8271d8 data, size: 36 bytes, Free: No
Block 0x3f827200 data, size: 12 bytes, Free: No
Block 0x3f827210 data, size: 40 bytes, Free: No 
Block 0x3f82723c data, size: 12 bytes, Free: No
Block 0x3f82724c data, size: 52 bytes, Free: No
Block 0x3f827284 data, size: 12 bytes, Free: No
Block 0x3f827294 data, size: 24 bytes, Free: Yes
Block 0x3f8272b0 data, size: 168 bytes, Free: No 
Block 0x3f82735c data, size: 168 bytes, Free: No
Block 0x3f827408 data, size: 12 bytes, Free: No
Block 0x3f827418 data, size: 1276 bytes, Free: No -> ?
Block 0x3f827918 data, size: 1224 bytes, Free: Yes -> ?
Block 0x3f827de4 data, size: 168 bytes, Free: No -> ?
Block 0x3f827e90 data, size: 168 bytes, Free: No
Block 0x3f827f3c data, size: 168 bytes, Free: No 
Block 0x3f827fe8 data, size: 168 bytes, Free: No
Block 0x3f828094 data, size: 168 bytes, Free: No
Block 0x3f828140 data, size: 168 bytes, Free: No
Block 0x3f8281ec data, size: 168 bytes, Free: No
Block 0x3f828298 data, size: 168 bytes, Free: No -> ?
Block 0x3f828344 data, size: 1684 bytes, Free: No
Block 0x3f8289dc data, size: 1920 bytes, Free: No -> From here my application
Block 0x3f829160 data, size: 10712 bytes, Free: No
Block 0x3f82bb3c data, size: 1920 bytes, Free: No 
Block 0x3f82c2c0 data, size: 492 bytes, Free: Yes -> ?
Block 0x3f82c4b0 data, size: 24576 bytes, Free: No
Block 0x3f8324b4 data, size: 921604 bytes, Free: No
Block 0x3f9134bc data, size: 128004 bytes, Free: No
Block 0x3f9328c4 data, size: 226968 bytes, Free: No 

I assume these smaller fragments are not related to wifi component anymore but to LWIP?

In sum the actual proposed workaround has two drawbacks:

• Global modification necessary -> project specific modification would be prefered
• Still SPIRAM fragementation (now running for a longer period but still not usable for productive environment)

Any further suggestions? Thanks in advance.

[mhdong]

Hi @Slider0007 Maybe you should also change here https://github.com/espressif/esp-idf/blob/d4cd437ede613fffacc06ac6d6c93a083829022f/components/lwip/port/include/lwipopts.h#L1636C1-L1647C50 `/**

• If CONFIG_ALLOC_MEMORY_IN_SPIRAM_FIRST is enabled, Try to
• allocate memory for lwip in SPIRAM firstly. If failed, try to allocate
• internal memory then. / #if CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP #define mem_clib_malloc(size) heap_caps_malloc_prefer(size, 2, MALLOC_CAP_DEFAULT|MALLOC_CAP_SPIRAM, MALLOC_CAP_DEFAULT|MALLOC_CAP_INTERNAL) #define mem_clib_calloc(n, size) heap_caps_calloc_prefer(n, size, 2, MALLOC_CAP_DEFAULT|MALLOC_CAP_SPIRAM, MALLOC_CAP_DEFAULT|MALLOC_CAP_INTERNAL) #else / !CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP / #define mem_clib_malloc malloc #define mem_clib_calloc calloc #endif / CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP */`

[Slider0007]

Hi @mhdong

thanks for your tip. Redirecting all allocations of LWIP component to internal memory even the flag CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP is set fragmentation of SPIRAM is under control and free internal memory is still a little larger than before (static allocs of WIFI component are now in SPIRAM). That's great.

But I'm still struggling to implement these changes on project level in a smart way. Right now all changes are made on framework level. How could I implement these on project level?

Is there any chance that a better configurability of the allocation strategy of the WIFI and LWIP component will be officially supported by the framework in the future to avoid such custom modifications?

[mhdong]

Hi @Slider0007 Thank you very much for your suggestion. Sure, we will try to optimize the allocation strategy of the WIFI and LWIP components.