How is memory allocated?

[butaikis]

Hi @bblanchon. I have ESP32-WROVER-IE (FLASH 8MB, PSRAM 8MB). I tested different amounts of memory allocation for JSON

DynamicJsonDocument testjs{7000000}; or DynamicJsonDocument testjs{70000000};

In all cases, the test js.capacity() function returned the correct number of bytes. But I don't understand how, because I don't have that much RAM.

Code:

  OUTPUT_JSON["buf_testjs"] = testjs.capacity();
  OUTPUT_JSON["psram_size_all"] = esp_spiram_get_size();
  OUTPUT_JSON["psram_size"] = ESP.getPsramSize();
  OUTPUT_JSON["psram_free"] = ESP.getFreePsram();
  OUTPUT_JSON["free_heap"]  = ESP.getFreeHeap();
  OUTPUT_JSON["free_heap2"]  = heap_caps_get_free_size(MALLOC_CAP_8BIT);
  OUTPUT_JSON["psram_himem_size"]  = esp_himem_get_phys_size();

Result:

  "buf_testjs": 70000000,
  "psram_size_all": 8388608,
  "psram_size": 3913347,
  "psram_free": 3791467,
  "free_heap": 55187,
  "free_heap2": 3809434,
  "psram_himem_size": 4456448

[bblanchon]

Hi @butaikis,

DynamicJsonDocument is a class from ArduinoJson 6. In ArduinoJson 7, you use JsonDocument without specifying the capacity. See https://arduinojson.org/v7/how-to/upgrade-from-v6/

Best Regards, Benoit

[butaikis]

@bblanchon I understand. But how do I determine that I have enough RAM for JSON? In the case of DynamicJsonDocument , we could check whether the required amount of memory was allocated or not.

If we use JsonDocument, we can use the function https://arduinojson.org/v6/api/jsondocument/overflowed/

but as indicated on the page,

true means that the memory pool was too small, so some values are missing in JsonDocument.

What does it mean that some values are missing? will there be no data at all or will something be cut out?

[bblanchon]

For example, if your object was initially {"first_name":"Lisa"}, and adding the member "last_name" fails, you could end up with either {"first_name":"Lisa"} or {"first_name":"Lisa","last_name":null} depending on the situation.

By the way, you are pointing to the documentation for version 6; which version are you using?

[butaikis]

@bblanchon sorry, I didn't fix the link to v7. I realized that the json itself remains intact, just the values are cut out. But it would be good to control RAM memory usage)

[butaikis]

@bblanchon Hello. Maybe there is a more efficient way to give a JSON response in the EspAsyncWebServer library?

At the moment, I have implemented it like this. (no more than 61440 bytes)

void WebServer_Response(JsonDocument &OUT_JSON, AsyncWebServerRequest *request)
{
  AsyncResponseStream *response = request->beginResponseStream("application/json", 61440);
  serializeJson(OUT_JSON, *response);
  request->send(response);
}

[bblanchon]

I'm not familiar with this library, but I'm surprised to see a hard-coded size here. I imagine the size you pass to beginResponseStream() will end up in the Content-Length header, so it has to be the same as the body's length. If that's the case, you should replace this value with measureJson(OUT_JSON).

[butaikis]

@bblanchon Yes, unfortunately, the author has not updated this library for a long time, although it is good. But there are some problems with sending big data. The second parameter in beginResponseStream specifies the size buffer using new cbuf[size]