ioBroker.js-controller icon indicating copy to clipboard operation
ioBroker.js-controller copied to clipboard

Backup failed with "Invalid string length" or out of memory

Open irudi opened this issue 2 years ago • 1 comments

I have a lot of objects and states (but that's another problem ...) - my states.jsonl is sized about 700MB.

If I try to backup, I get:

# iob backup host.iobroker 68291 states saved host.iobroker 31599 objects saved host.iobroker Backup not created: Invalid string length Backup created: undefined

A similar problem is reported in https://forum.iobroker.net/post/718012

I think the problem is the JSON.stringify function in setupBackup.ts (line 432). This function has problems with large input (as noted in #1717 as well).

There are some ideas to overcome this problem in https://stackoverflow.com/questions/29175877/json-stringify-throws-rangeerror-invalid-string-length-for-huge-objects

irudi avatar Nov 09 '22 06:11 irudi

Maybe the idea to really do like stackoverflow had as idea (aka "store each key and build the highest stringify level yourself as string actions) - but then we could also prevent having a bit string in memory when doing backup and iteratively write that to the file ... could also reduce mem footprint of backup

Apollon77 avatar Nov 09 '22 08:11 Apollon77