react-native-mmkv
react-native-mmkv copied to clipboard
mrousavy
MMKV storage file grows until app crash
The library is great but one crucial issue doesn't give me chance to use it.
And it grows by stop 64Mb, 128Mb etc. If I use slow FS storage, the file size never becomes more than 15Mb. I also used the console log to check the size of the string.
It's working really fast and I want to use that, but with every rewriting, it becomes more significant. I checked key is always stable.
"react-native-mmkv": "^2.4.3", "react-native": "0.66.3",
How often are you writing to MMKV? It's not a database, it's a memory map key value storage.
How often are you writing to MMKV? It's not a database, it's a memory map key value storage.
I'm writing when my state change with a few seconds of throttling. I use it with redux-persist. But why does the size of the file grow? As I mentioned before, I use FS storage, which is significantly slower, but the file size increases up to 15Mb. I use only one key.
i have the same "bug"
Check the explanation here https://github.com/ammarahm-ed/react-native-mmkv-storage/issues/286
Seems that the Key Value doesn't overwrite the value, just add a new row into the storage. @mrousavy
Please @ryskin let me know if you solved the problem
Possibly related? https://github.com/mrousavy/react-native-mmkv/issues/397
@mrousavy to clear things up, does mmkv insert a new row for every setter, without overwriting the previous value? If so, how can we properly clear 'old' entries?
@aLemonFox have you tried removing the previous entry before setting the new one?
@mrousavy Confirming this contributes to my app crashes. The memory is hogging in Android 12 Go edition which typically 2GB or less ram.
@shawarmaz this is dangerous because you can get into two cases:
1 -> async delete, then async write -> maybe write is too slow so there is no advantage to using mmkv 2 -> race condition -> The new data can be deleted
I see the point on how this memory is designed but should have some cleaning method to reduce the memory consumption, there is no point in keeping dozens of versions (Is there any method to recover previous versions? I think that no, so it is pointless)
@ramirobg94 neither of those cases apply because the calls are sync. To be clear that means not async.
Checking if it exists and deleting before set seems to have worked; I've used patch-package, but I can open a pr with the fix:
diff --git a/node_modules/react-native-mmkv/lib/module/MMKV.js b/node_modules/react-native-mmkv/lib/module/MMKV.js
index 8856aea..9cbe764 100644
--- a/node_modules/react-native-mmkv/lib/module/MMKV.js
+++ b/node_modules/react-native-mmkv/lib/module/MMKV.js
@@ -45,6 +45,12 @@ export class MMKV {
}
set(key, value) {
const func = this.getFunctionFromCache('set');
+ const contains = this.getFunctionFromCache('contains');
+ if (key && contains(key)) {
+ const _delete = this.getFunctionFromCache('delete');
+ _delete(key);
+ }
func(key, value);
this.onValuesChanged([key]);
}
You could also just leave the delete call there before set to avoid increasing the time this function takes to finish:
diff --git a/node_modules/react-native-mmkv/lib/module/MMKV.js b/node_modules/react-native-mmkv/lib/module/MMKV.js
index 8856aea..11b43da 100644
--- a/node_modules/react-native-mmkv/lib/module/MMKV.js
+++ b/node_modules/react-native-mmkv/lib/module/MMKV.js
@@ -45,6 +45,10 @@ export class MMKV {
}
set(key, value) {
const func = this.getFunctionFromCache('set');
+ if (key) {
+ const _delete = this.getFunctionFromCache('delete');
+ _delete(key);
+ }
func(key, value);
this.onValuesChanged([key]);
}
I don’t think this should be merged, maybe just a doc update. You can just wrap the method yourself.
I don't think this should be left as is, having it fill up memory is a serious issue, having it go a little slower is much better than having it crash your app.
Again, this is how mmkv is designed. You can wrap the method yourself. Wrapping it lib-level would give new non-mmkv semantics to the lib. To keep it 💯 mmkv, the actual solution to this issue is mmkv’s trim method.
@mrousavy , what was the issue with this PR?
https://github.com/mrousavy/react-native-mmkv/pull/461
I don't know how not managing space allocation could be by design? Managing memory/space allocation is the libs responsibility, otherwise it's a insta leak to unaware devs. There should be a big warning in the readme, a temporary fix like that can be given but the way data is stored needs to be changed to fix the leak, or at least trim should be exposed and it could be called in specific situations, even though this delegates responsibility to devs and they can't really tell when trim should be called because they can't see how much space has been allocated.
So it's either delete and trim manually or move to another lib like react-native-mmkv-storage, gotcha. Another thing is that it seems like trim isn't a exposed method
I don't know, I started to use other MMKV libraries and problem disappear, file is small and works like charm
I don't know, I started to use other MMKV libraries and problem disappear, file is small and works like charm
which lib？
I'm thinking not good to post links to other libraries here, but you can google only 2 main libraries available for react-native
I don't know, I started to use other MMKV libraries and problem disappear, file is small and works like charm
which lib？
react-native-mmkv-storage works just fine
@mrousavy , what was the issue with this PR? https://github.com/mrousavy/react-native-mmkv/pull/461
@shawarmaz I found out that printing the total size after doing a trim made no difference, so I wasn't able to reproduce your error. I could take another look, but we wanna find the right solution here without sacrificing speed :)
Can you guys create a reproduceable example? Is this on Android only, or iOS too?
Personally I just wrapped writes with preprended deletes and called it a day. No noticeable difference in write performance.
If you're testing against high volume of writes or high write size, you're not using the right tool for the job. The official Tencent/MMKV repo benchmarks it against NSUserDefaults/SharedUserPreferences which tells you all you need to know about its intended usage i.e. not as a local runtime database.
IMO the main selling point of MMKV is not speed but having a unified API for use cases that would have otherwise required NSUserDefaults/SharedUserPreferences.
If you want to store application data like chat messages or POI geopoints etc, use the right tool for the job i.e. something else. Even if trim was available everybody would be like wtf why do I need to do this. It's counterintuitive because you think MMKV is something that it is not.
This may be a case of false advertisement around MMKV at large. Thoughts welcome
@shawarmaz I think you're right. I'm struggling with this too. Are there any other good reactive key value store libraries available which are not based on this? I can't seem to find any. They are either like AsyncStorage (async, not reactive) or WatermelonDB (full db, want key value store like Redis).
@aLemonFox just use this lib and wrap writes with a preprended delete
setItem: (key, value) => {
storage.delete(key)
storage.set(key, value);
},
...or just unsexy AsyncStorage whose writes do overwrite and that also does not load entire data from cold storage in memory, unexpectedly
Which in fact also illustrates that comparisons/benchmarks between AsyncStorage and MMKV are inherently false/misleading.
@aLemonFox Btw mmm MMKV is not reactive
Personally I just wrapped writes with preprended deletes and called it a day. No noticeable difference in write performance.
If you're testing against high volume of writes or high write size, you're not using the right tool for the job. The official Tencent/MMKV repo benchmarks it against
NSUserDefaults/SharedUserPreferenceswhich tells you all you need to know about its intended usage i.e. not as a local runtime database.IMO the main selling point of MMKV is not speed but having a unified API for use cases that would have otherwise required NSUserDefaults/SharedUserPreferences.
If you want to store application data like chat messages or POI geopoints etc, use the right tool for the job i.e. something else. Even if
trimwas available everybody would be like wtf why do I need to do this. It's counterintuitive because you think MMKV is something that it is not.This may be a case of false advertisement around MMKV at large. Thoughts welcome
It's not only falsely advertised, it's broken by design. Moving from async to mmkv is like switching to a bmw but that suddenly crashes into a lamp post if you push it too hard.
Anyway, we need to test to be sure that prepending delete to write calls does actually fix the leak.
@aLemonFox Btw mmm MMKV is not reactive
I think local storage should not be reactive, for that it's better to actually use something like Zustand or Redux and persist that data on specific moments, either with a specific solution or something like mmkv
It's not only falsely advertised, it's broken by design. Moving from async to mmkv is like switching to a bmw but that suddenly crashes into a lamp post if you push it too hard.
Anyway, we need to test to be sure that prepending delete to write calls does actually fix the leak.
@arthurgeron No it's not broken by design, it is your understanding of its design that is broken. It is extremely efficient at what it is designed for, surely everyone touting its benefits aren't idiots. Did you even read my reply you quoted? Going from AsyncStorage to MMKV when you have a big load of high-write data is like trying to park a schoolbus in your garage.
I think local storage should not be reactive, for that it's better to actually use something like Zustand or Redux and persist that data on specific moments, either with a specific solution or something like mmkv
Read the context before replying please