Use IndexedDB rows instead of "flat" data

[Irrelon]

Currently the DB stores data "flat" because we need compatibility with localStorage. Make two storage code paths depending on if we are storing via localStorage (flat string data) or IndexedDB (row-based storage).

[urossmerdelj]

Don't mean to hijack this thread but will there be a change-log available anywhere, especially since there was a major update released today from 1.4.x to 2.0.x?

I'm particularly interested in any changes to persistence - it is currently a major resource hog. The issues we're experiencing at the moment is persisting large collections (>5000 documents per collection and with IDB size > 30MB). Doing a db.save in the above example can bump memory usage by 100MB and most of that memory isn't GC-ed for quite a while (~2/3 is GC-ed after 10-20s but not all - meaning memory footprint raises constantly by doing a reasonably eager persisting). We eagerly persist to IDB by debouncing (5s) collection.save on CRUD operations.

The main culprit behind resource hog seems to be JSON.parse/JSON.stringify as also noted on stackoverflow: https://stackoverflow.com/questions/30564728/json-parse-on-a-large-array-of-objects-is-using-way-more-memory-than-it-should

[dmccarrick]

Any news on this, yet?

[Irrelon]

Hi everyone. I am very swamped, finding it impossible to get time to apply these sorts of changes at the moment. Any help would be much appreciated!!

If you have the technical inclination to attempt this update please let me know. I will endeavour to provide details for any questions you have if you wish to try and make a PR for this update.

[no-more]

Hi Irrelon,

I'd love to help but currently I'm stuck too. If I manage to get some free time later this year (not soon) I will try to help. This is an amazing project and I hope we'll see planned features implemented.

Thanks for all your work.

[Irrelon]

Thanks so much guys. I reckon this update would be a major positive change.