electron-store icon indicating copy to clipboard operation
electron-store copied to clipboard

Maximum recommended data size?

Open kiran-peddikuppa opened this issue 6 years ago • 7 comments

What is the maximum file size we can store in this. Can i store 100MB file

kiran-peddikuppa avatar Jan 05 '19 11:01 kiran-peddikuppa

That's not a good idea as everything is currently saved into a single JSON file and needs to be parsed for each operation. Would be interesting to explore a way to have file references, so the JSON file only had a reference to the file which could be stored separately. Maybe it could automatically save it as a separate file if the data is a Buffer/TypedArray.

However, you can already do manual file references yourself by saving the file and then storing the file path in electron-store.

Just curious, what kind of data do you intend to store? What format? Buffers, Base64, etc?

sindresorhus avatar Jan 05 '19 16:01 sindresorhus

I am interested to know also. I will be storing data based on user settings and race track configurations. Should they all be in separate files? Can this be easily done?

callen5914 avatar Mar 16 '19 12:03 callen5914

I personally have begun using this project to cache Base64 Encoded image renders. I've found it really begins to slow down around the 100mb mark, especially when each entry is very large (30mb each). I will experiment more as I integrate the project further.

stuible avatar Mar 25 '19 23:03 stuible

Just to follow up, I decided that this was a very bad idea and was not performant at all. When you approach 50mb, performance is noticeably effected. The easy workaround was to store the large files as separate files using node fs and then store a record of them in electron-store.

stuible avatar Apr 13 '19 02:04 stuible

Can electron-store offer options store files separately? It would be nice if implemented

harryzcy avatar Jun 12 '19 12:06 harryzcy

Can electron-store offer options store files separately? It would be nice if implemented

electron-store does support multiple separately files. See at options -> name & cwd:

https://github.com/sindresorhus/electron-store#name https://github.com/sindresorhus/electron-store#cwd

YanDevDe avatar Sep 01 '19 16:09 YanDevDe

Just to follow up, I decided that this was a very bad idea and was not performant at all. When you approach 50mb, performance is noticeably effected. The easy workaround was to store the large files as separate files using node fs and then store a record of them in electron-store.

Thank you for your try. My plan is to store some about 10mb cache files, obvioisly, my concern is unnecessary. I will tell here if there are any performance problems.

imxuedi avatar Jan 26 '23 13:01 imxuedi