bblanchon
[v7] `CappedAllocator` for ArduinoJson 7
The switch to a dynamically allocated buffer bothers us because we have pretty big use case (streaming) where functions are populating a json document to be sent to Web Socket.
The problem is that the whole Json document can take up to 16-32k so we have to split it in batches sent consecutively.
So we are relying on the capacity of a Json document to know when it overflows, to send the current json document in the web socket channel, then clear the document and continue the loop so that the application can continue populating the next json objects in the document.
This "streaming" approach allows us to limit the memory usage while not knowing in advance the complete size of the document and number of batches.
Switching to v7 does not allow such approach: we would need to re-implement an allocator, but the current API is incomplete for the reallocate and deallocate parts: we were thinking to do something like that:
namespace ArduinoJson {
class CappedAllocator : Allocator {
public:
explicit CappedAllocator(size_t capacity, Allocator *delegate = detail::DefaultAllocator::instance()) : _capacity(capacity), _delegate(delegate) {}
void *allocate(size_t size) {
if (_allocated + size > _capacity)
return nullptr;
void *p = _delegate->allocate(size);
if (p)
_allocated += size;
return p;
}
void deallocate(void *pointer, size_t size) {
_delegate->deallocate(pointer, size);
_allocated -= size;
}
void *reallocate(void *pointer, size_t old_size, size_t new_size) {
void *p = _delegate->reallocate(pointer, old_size);
_allocated -= old_size;
if (p)
_allocated += new_size;
return p;
}
private:
const size_t _capacity;
Allocator *_delegate;
size_t _allocated = 0;
};
}
What we would need is something like this in order to keep track of the capacity:
virtual void deallocate(void* ptr, size_t size) = 0;
virtual void* reallocate(void* ptr, size_t old_size, size_t new_size) = 0;
Looking at the ArduinoJson code base, each time allocate is called, there is a variable on caller side for the allocated capacity, so it should be possible to provide such interfaces, and even keep backward compatibility by providing stubs.
What do you think ?
The current unacceptable and slow alternative I see would be to call measureJson() instead of overflowed(), and make sure the json document has enough capacity, and we send the batches once our json document reaches a size limit.
Is there otherwise another (fast) way to implement such steaming approach ?
Thanks a lot for this awesome library :-)
I've tried a temporary implementation with a map keeping tack of the allocated sizes per pointer but it does not work.
Impl:
class ESPDashAllocator : public ArduinoJson::Allocator {
public:
explicit ESPDashAllocator(size_t capacity, ArduinoJson::Allocator *delegate = ArduinoJson::detail::DefaultAllocator::instance()) : _capacity(capacity), _delegate(delegate) {}
void *allocate(size_t size) {
if (_allocated + size > _capacity)
return nullptr;
void *p = _delegate->allocate(size);
if (p) {
_allocations[p] = size;
_allocated += size;
}
return p;
}
void deallocate(void *pointer) {
_delegate->deallocate(pointer);
auto it = _allocations.find(pointer);
if (it != _allocations.end()) {
_allocated -= it->second;
_allocations.erase(it);
}
}
void *reallocate(void *pointer, size_t new_size) {
void *p = _delegate->reallocate(pointer, new_size);
if (p) {
auto it = _allocations.find(pointer);
if (it != _allocations.end()) {
_allocated -= it->second;
_allocations.erase(it);
}
_allocations[p] = new_size;
_allocated += new_size;
}
return p;
}
size_t capacity() { return _capacity; }
size_t allocated() { return _allocated; }
size_t blocks() { return _allocations.size(); }
private:
const size_t _capacity;
Allocator *_delegate;
size_t _allocated = 0;
std::map<void *, size_t> _allocations;
};
I have the following crash below. We can see that the 4 first websocket messages are sent: the allocator is configured with a max capacity of 2k, when the allocator returns null, overflowed is triggered and packets are sent and the JsonDocument is cleared and filled by following calls. But it crashes after a few rounds.
T [DASH] client=1, capacity=2048, allocated=1043, measureJson=612
T [DASH] client=1, capacity=2048, allocated=1138, measureJson=688
T [DASH] client=1, capacity=2048, allocated=1090, measureJson=642
T [DASH] client=1, capacity=2048, allocated=1050, measureJson=567
assert failed: ArduinoJson::V701PB2::detail::VariantSlot* ArduinoJson::V701PB2::detail::VariantPoolList::getSlot(ArduinoJson::V701PB2::detail::SlotId) const VariantPoolList.hpp:98 (poolIndex < count_
Backtrace: 0x400838e5:0x3ffe9ad0 0x4008dbcd:0x3ffe9af0 0x40093719:0x3ffe9b10 0x400dd1ec:0x3ffe9c40 0x4012437f:0x3ffe9ce0 0x4012676f:0x3ffe9d00 0x40128a1a:0x3ffea200 0x40128f3d:0x3ffea280 0x401c59b1:0x3ffea2b0 0x401c59d6:0x3ffea300 0x401c62b1:0x3ffea330 0x401c6425:0x3ffea370 0x401c2f2b:0x3ffea390 0x401c2f7a:0x3ffea3c0 0x401c2fbd:0x3ffea3e0 0x401c3196:0x3ffea400 0x401c3209:0x3ffea420
#0 0x400838e5:0x3ffe9ad0 in panic_abort at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/esp_system/panic.c:408
#1 0x4008dbcd:0x3ffe9af0 in esp_system_abort at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/esp_system/esp_system.c:137
#2 0x40093719:0x3ffe9b10 in __assert_func at /Users/ficeto/Desktop/ESP32/ESP32S2/esp-idf-public/components/newlib/assert.c:85
#3 0x400dd1ec:0x3ffe9c40 in ArduinoJson::V701PB2::detail::VariantPoolList::getSlot(unsigned short) const at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Memory/VariantPoolList.hpp:98
(inlined by) ArduinoJson::V701PB2::detail::VariantPoolList::allocFromFreeList() at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Memory/VariantPoolImpl.hpp:71
(inlined by) ArduinoJson::V701PB2::detail::VariantPoolList::allocSlot(ArduinoJson::V701PB2::Allocator*) at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Memory/VariantPoolList.hpp:73
(inlined by) ArduinoJson::V701PB2::detail::ResourceManager::allocSlot() at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Memory/ResourceManager.hpp:53
(inlined by) ArduinoJson::V701PB2::detail::CollectionData::addSlot(ArduinoJson::V701PB2::detail::ResourceManager*) at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Collection/CollectionImpl.hpp:52
(inlined by) ArduinoJson::V701PB2::detail::VariantData* ArduinoJson::V701PB2::detail::ObjectData::addMember<ArduinoJson::V701PB2::detail::StaticStringAdapter>(ArduinoJson::V701PB2::detail::StaticStringAdapter, ArduinoJson::V701PB2::detail::ResourceManager*) at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Object/ObjectData.hpp:27
(inlined by) ArduinoJson::V701PB2::detail::VariantData* ArduinoJson::V701PB2::detail::ObjectData::getOrAddMember<ArduinoJson::V701PB2::detail::StaticStringAdapter>(ArduinoJson::V701PB2::detail::StaticStringAdapter, ArduinoJson::V701PB2::detail::ResourceManager*) at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Object/ObjectImpl.hpp:24
#4 0x4012437f:0x3ffe9ce0 in ArduinoJson::V701PB2::detail::VariantData* ArduinoJson::V701PB2::detail::VariantData::getOrAddMember<ArduinoJson::V701PB2::detail::StaticStringAdapter>(ArduinoJson::V701PB2::detail::StaticStringAdapter, ArduinoJson::V701PB2::detail::ResourceManager*) at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Variant/VariantData.hpp:226
#5 0x4012676f:0x3ffe9d00 in ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*>::getOrCreateData() const at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Object/MemberProxy.hpp:59
(inlined by) ArduinoJson::V701PB2::detail::VariantData* ArduinoJson::V701PB2::detail::VariantAttorney::getOrCreateData<ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*> const>(ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*> const&) at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Variant/VariantAttorney.hpp:32
(inlined by) ArduinoJson::V701PB2::detail::VariantRefBase<ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*> >::getOrCreateData() const at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Variant/VariantRefBase.hpp:268
(inlined by) ArduinoJson::V701PB2::detail::VariantRefBase<ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*> >::getOrCreateVariant() const at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Variant/VariantRefBaseImpl.hpp:100
(inlined by) bool ArduinoJson::V701PB2::detail::VariantRefBase<ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*> >::set<char const>(char const*) const at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Variant/VariantRefBaseImpl.hpp:144
(inlined by) ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*>& ArduinoJson::V701PB2::detail::MemberProxy<ArduinoJson::V701PB2::JsonDocument&, char const*>::operator=<char const>(char const*) at .pio/libdeps/pro-esp32-debug/ArduinoJson/src/ArduinoJson/Object/MemberProxy.hpp:39
(inlined by) ESPDash::generateLayoutJSON(AsyncWebSocketClient*, bool, Card*) at lib/ESPDASHPro/src/ESPDashPro.cpp:353
#6 0x40128a1a:0x3ffea200 in ESPDash::ESPDash(AsyncWebServer*, char const*, bool)::{lambda(AsyncWebSocket*, AsyncWebSocketClient*, AwsEventType, void*, unsigned char*, unsigned int)#3}::operator()(AsyncWebSocket*, AsyncWebSocketClient*, AwsEventType, void*, unsigned char*, unsigned int) const at lib/ESPDASHPro/src/ESPDashPro.cpp:87
#7 0x40128f3d:0x3ffea280 in std::_Function_handler<void (AsyncWebSocket*, AsyncWebSocketClient*, AwsEventType, void*, unsigned char*, unsigned int), ESPDash::ESPDash(AsyncWebServer*, char const*, bool)::{lambda(AsyncWebSocket*, AsyncWebSocketClient*, AwsEventType, void*, unsigned char*, unsigned int)#3}>::_M_invoke(std::_Any_data const&, AsyncWebSocket*&&, AsyncWebSocketClient*&&, AwsEventType&&, void*&&, unsigned char*&&, unsigned int&&) at /Users/mat/.platformio/packages/[email protected]+2021r2-patch5/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:297
#8 0x401c59b1:0x3ffea2b0 in std::function<void (AsyncWebSocket*, AsyncWebSocketClient*, AwsEventType, void*, unsigned char*, unsigned int)>::operator()(AsyncWebSocket*, AsyncWebSocketClient*, AwsEventType, void*, unsigned char*, unsigned int) const at /Users/mat/.platformio/packages/[email protected]+2021r2-patch5/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:687
#9 0x401c59d6:0x3ffea300 in AsyncWebSocket::_handleEvent(AsyncWebSocketClient*, AwsEventType, void*, unsigned char*, unsigned int) at .pio/libdeps/pro-esp32-debug/ESPAsyncWebServer-esphome/src/AsyncWebSocket.cpp:862
#10 0x401c62b1:0x3ffea330 in AsyncWebSocketClient::_onData(void*, unsigned int) at .pio/libdeps/pro-esp32-debug/ESPAsyncWebServer-esphome/src/AsyncWebSocket.cpp:682
#11 0x401c6425:0x3ffea370 in std::_Function_handler<void (void*, AsyncClient*, void*, unsigned int), AsyncWebSocketClient::AsyncWebSocketClient(AsyncWebServerRequest*, AsyncWebSocket*)::{lambda(void*, AsyncClient*, void*, unsigned int)#7}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&, std::_Any_data const&, unsigned int&&) at .pio/libdeps/pro-esp32-debug/ESPAsyncWebServer-esphome/src/AsyncWebSocket.cpp:481
(inlined by) _M_invoke at /Users/mat/.platformio/packages/[email protected]+2021r2-patch5/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:297
#12 0x401c2f2b:0x3ffea390 in std::function<void (void*, AsyncClient*, void*, unsigned int)>::operator()(void*, AsyncClient*, void*, unsigned int) const at /Users/mat/.platformio/packages/[email protected]+2021r2-patch5/xtensa-esp32-elf/include/c++/8.4.0/bits/std_function.h:687
#13 0x401c2f7a:0x3ffea3c0 in AsyncClient::_recv(tcp_pcb*, pbuf*, signed char) at .pio/libdeps/pro-esp32-debug/AsyncTCP-esphome/src/AsyncTCP.cpp:961
#14 0x401c2fbd:0x3ffea3e0 in AsyncClient::_s_recv(void*, tcp_pcb*, pbuf*, signed char) at .pio/libdeps/pro-esp32-debug/AsyncTCP-esphome/src/AsyncTCP.cpp:1253
#15 0x401c3196:0x3ffea400 in _handle_async_event(lwip_event_packet_t*) at .pio/libdeps/pro-esp32-debug/AsyncTCP-esphome/src/AsyncTCP.cpp:164
#16 0x401c3209:0x3ffea420 in _async_service_task(void*) at .pio/libdeps/pro-esp32-debug/AsyncTCP-esphome/src/AsyncTCP.cpp:199
ELF file SHA256: 93e73deb7c556b58
E (8166) esp_core_dump_flash: Core dump flash config is corrupted! CRC=0x7bd5c66f instead of 0x0
Rebooting...
The line crashing is a simple assignment like this one: doc["command"] = changes_only ? "update:components" : "update:layout:next"; but I think this is unrelated since it crashes even before trying to allocate a slot if I understand, so teh allocator is not even involved.
I gave up on the allocator and implemented the measureJson way.
T [DASH] client=2, measureJson=612
T [DASH] client=2, measureJson=1047
T [DASH] client=2, measureJson=1067
T [DASH] client=2, measureJson=1031
T [DASH] client=2, measureJson=1078
T [DASH] client=2, measureJson=1063
T [DASH] client=2, measureJson=1040
T [DASH] client=2, measureJson=1069
T [DASH] client=2, measureJson=1082
T [DASH] client=2, measureJson=1079
T [DASH] client=2, measureJson=1059
T [DASH] client=2, measureJson=1050
T [DASH] client=2, measureJson=1030
T [DASH] client=2, measureJson=824
T [DASH] client=2, measureJson=1040
T [DASH] client=2, measureJson=159
Using measureJson increases the publish time to websocket by a few ms so this is acceptable for a big use case like that.
Hi @mathieucarbou,
I often considered changing the allocator interface to include the old size as in std::allocator, but I never found a real use case. I'm afraid "relying on the capacity of a Json document to know when it overflows, to send the current json document" isn't a strong use case either.
Using measureJson increases the publish time to websocket by a few ms so this is acceptable for a big use case like that.
The JSON document is fairly small. Why don't you use a buffer and send it when it reaches a certain level?
char buffer[2048];
size_t len = serializeJson(doc, buffer);
if (len > 1024) {
sendJson(buffer, len);
doc.clear();
}
Best regards, Benoit
Actually, I am serialising into AsyncWebSocketMessageBuffer* buffer = _ws->makeBuffer(size); to avoid the memory copy happening (that's why I am measuring). That's always a tradeoff between speed and memory. I will try allocating a buffer like that and see what happens. Thanks for the suggestion!