Saving 170k+ entries crashes process

[eL1x00r]

Any idea on how to increase performance for bigger scale?

[marcus-pousette]

Hello!

I have planned to fix this issue coming 2 weeks. But basically this issue is about bounding RAM usage. I am working on a video on-demand solution with Peerbit and I am starting to see similiar issues when videos are long.

Two things to fix

1. Basically (I am assuming you are using the Document store). Is that the index is now in memory, but needs to be on disc when there are many documents. One "easy" solution is the change out the index "engine" to SQLLite with OPFS storage and that problem would be solved.

2. For the syncing engine, the hashes of all "heads" are kept in memory. This will eventually also create a problem that is similar to (1) and needs to be solved in a similar way as (1). The RAM usage due to this is most likely lower than for the document index.

[eL1x00r]

fire, keep me updated!