realm-js
realm-js copied to clipboard
realm
Stability of Realm - More Open Communication
Hey! I really appreciate how you are constantly improving Realm and pushing new updates!
However, lately, I got a feeling that the stability of Realm has decreased to a state where it feels too risky to update Realm to the latest version. I want to clarify that this is a feeling, so this might not be a reality. I believe this could be fixed by more open communication about critical bugs.
I think the first major issue happened with v10.12.0. The update bumped the Realm Core version into v11.8.0. Unfortunately, the Core version contained a bug that if you have enabled encryption, unhandled crashes would start to occur. If you were unlucky, the crashes would corrupt the entire database. In other words, if v10.12 was used in the production, the customers might have lost all their data.
The issue was fixed with version v10.16.0. The versions v10.13 - v10.15 were released without mentioning the critical bug at all. Finding the critical bug from the open issues was really difficult since there were over 300 open issues, and the meta issue was actually even located under the Realm Core repository instead of this one. So, if you weren't reading the issues on a weekly basis, you wouldn't have any knowledge about versions v10.12 - v10.15 being broken.
At the moment, there are more open issues that might be critical. For instance, this https://github.com/realm/realm-js/issues/4525 sounds like a critical issue that could make the app crash randomly. However, I have no clue about whether it's actually a critical bug or not. And I have no idea whether there are more bugs like that but I just cannot find them from the issue list.
In other words, the issue is that it's unclear whether there are open critical issues or not. It's stopping us, and probably many others also, from updating Realm since no one can risk that e.g. an app starts crashing randomly for thousands of customers.
Solution
I believe that this issue could be solved by more open communication. If there was a list of open critical bugs, it would help a lot. In addition, when releasing a new version that contains an existing critical bug, it would be really nice if the release documentation mentioned and warned about the bug.
The important and difficult thing is also to keep the list up to date. So, if there is some issue open for over 1 year, it's probably not a critical issue. For instance, if you search for the issues having the tag T-Bug-Crash, it's not really helping to identify critical crashing issues. There are open issues from the years 2017 and 2018, which probably means that the crashes are either really minor ones or the list is not up to date.
I would recommend adding a tag e.g. called Critical. I would classify something to be critical if the bug might corrupt the entire database or if the new Realm version starts to crash more often than the previous versions. However, I fully trust you can define the word "critical" better on your own.
Thank you for the great library and the time and effort you invest into maintaining it
Hey @Frans-L, thanks for the feedback! I'm going to raise the points you raised for discussion with our team and we will get back to you with our thoughts. Ultimately our goal is not to have any critical bugs, and there is a lot of work under way to improve testing and stability, but of course these things can slip through even with the best testing in the world, so it would be good for us to be clear about what to expect in these cases.
@Frans-L Sorry for not getting back earlier.
I fully understand the needs for you (not just @Frans-L but all of our users) to be able to follow the development of Realm JavaScript. We are truly thrilled to have engaged users. And we would love to learn about your app.
The severity of an issue can be difficult to asses. In some cases, we are not able to reproduce an issue, and it is easy to be left with the impression that the issue is rare. For the reporter, the issue is Critical while we might believe it is not.
We have been haunted by memory issues on iOS (examples: #4473 and #3602). We have - without luck - tried to reproduce these crashes. We put our investigations on hold a while back as the old JSC integration (which is on master currently) is scheduled to be replaced by our JSI/Hermes integration. Unfortunately, we have to delay JSI/Hermes a bit, and we need to return the investigation. #4525 is a mixture of different issues, and a fix in v10.14.0 solved it for new Realm files (existing Realm files might still be affected).
I agree that our guidelines around the fix in v10.16.0 hasn't been clear. We have a Known Issues section, and we added it since we received many questions about debugging. When we identify an issue (and it might be reported in one of our other SDKs) as one potentially causing corruption, I agree a simple label is a good solution. A question to the audience: when we fix a bug which potentially can cause corruption, where you do want to read about it and how to recover? In the associated issue(s)? Or CHANGELOG.md or README.md?
Hey @Frans-L I have these feelings too, but for me is about to know what version is stable to be used. I'm trying to use realm and @realm/react with a RN app (and Expo) for a couple of months but until now I didn't make it work properly.
I have faced 2 crashing bugs and when I work around the first one (that is reanimated 2 related), I had no clue what is going wrong with the second one, or how to work around it.
I want to say thank you for improving the stability.
After using Realm for 4 past years, Realm seems in the most stable position it has ever been.
I'll close this issue since I don't believe this is any more relevant to keep open :)
@Frans-L thanks a lot for the encouragement!
We definitely too feel the v12 of our SDK is much more stable, mainly because almost all of our C++ code is now generated, which results in a much more unified call pattern across surface of our SDK. We strongly encourage anyone still on v11 to upgrade, as we've spent great effort minimising breaking changes across this major version, although the SDK is pretty much re-written.
We are actively working on improving stability of the package, specifically related to ABI stability of our prebuild binaries: We expect this to lift restrictions on the compatibility between versions of Realm, React Native and other libraries using native modules.
@Frans-L We appreciate your kind words. As @kraenhansen says, the team has put a lot of effort into making the SDK more stable - and more pleasant to work on as an SDK developer.
The open communication will be important forever and we will try not to forget this issue 😄