Interprocess support for encrypted Realms

[KynoYang]

Realm version is 0.8.2, Android version is 4.4.2

Encrypted interprocess sharing is currently unsupported in io_realm_internal_SharedGroup.cpp line 115 at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(SharedGroup.java) at io.realm.internal.SharedGroup.(SharedGroup.java:60) at io.realm.Realm.(Realm.java:207) at io.realm.Realm.createAndValidate(Realm.java:597) at io.realm.Realm.create(Realm.java:567) at io.realm.Realm.getInstance(Realm.java:540)

Anyone know what it is?

[beeender]

Are you using same Realm in different processes? @KynoYang Multi-processes for java is not supported yet...

[kneth]

As @beeender points out, multiprocess access isn't yet supported - see #1091.

[beeender]

@KynoYang is still seeing this issue http://crashes.to/s/afa05a92642 on smartisan S701 This should be the dup issue to #2762

[phamducgiam]

Hi.

I encountered this issue too cause of using firebase. It causes by adding plugin 'com.google.gms.google-services' for automatically integrating firebase sdk. If I removed that plugin or remove realm encryption, everything is ok. Any suggesstion? Thanks.

[beeender]

@phamducgiam Do you have something like process:remote in your manifest?

[phamducgiam]

@beeender: I just found out. It's because firebase crash automatically create a background process for reporting. I just disable firebase crash for now and will fix it later.

[mvojtkovszky]

Here's my observation, reposted here as instructed in #3203

io.realm.exceptions.RealmError: Unrecoverable error. /data/data/<package>/files/database.realm: Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 22257. Current pid is 17958. in io_realm_internal_SharedGroup.cpp line 113
    at io.realm.internal.SharedGroup.createNativeWithImplicitTransactions(Native Method)
    at io.realm.internal.SharedGroup.a(SourceFile:95)
    at io.realm.internal.SharedGroup.<init>(SourceFile:74)
    at io.realm.internal.SharedGroupManager.<init>(SourceFile:49)
    at io.realm.BaseRealm.<init>(SourceFile:81)
    at io.realm.Realm.<init>(SourceFile:140)
    at io.realm.Realm.b(SourceFile:240)
    at io.realm.Realm.a(SourceFile:220)
    at bbq.a(SourceFile:126)
    at io.realm.Realm.getDefaultInstance(SourceFile:166)

Crashlytics reports this kind of crash for around 1,5% of about 50k users so far and I can't figure out what causes it, neither reproduce it on any of our test devices or emulator.

Things to consider:

• getDefaultInstance() that causes the crash is always called inside onCreate() method of our Application class. That's where Realm's default configuration gets set and default instance is obtained.
• Our manifest file doesn't define any remote processes, like process:remote
• If a third party library created a remote process, we would expect the crash to always happen? (I also try to catch it with RealmIOException, but that doesn't seem to do anything for any reason)
• Crash always spikes on the first days following the update and drops significantly after, indicating that only users who updated get affected.
• Considering the sheer amount of crashes and users affected, we get no complaint emails reporting this.

Version of Realm and tooling

Realm version(s): 1.1.0 Android Studio version: 2.1.2 Which Android version and device: all supported (4.0.3+)

[beeender]

@biftekman A hypothesis is there is a bug with some devices's package manager. When installing and running the new version of APK, the old process has not been killed yet -- This would trigger your issue and #2928 .

Crash always spikes on the first days following the update and drops significantly after, indicating that only users who updated get affected.

And I think above fact supports our hypothesis well.

We made some workaround try to solve #2928 -- delay some time if we see there is another process alive and opening the Realm file, but from feedback, it doesn't help too much :(

[mvojtkovszky]

@beeender does the crash then kill the old process as well? Meaning that reopening the app afterwards would occur normally?

[beeender]

@biftekman It will only kill the process which throws this exception. But from the exception message:

Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 22257. Current pid is 17958. in io_realm_internal_SharedGroup.cpp

Interesting thing is we can see it is actually the old process throws the exception ( 17958, since the pid number is not big enough, i guess we can assume 22257 is started later than 17958).

Can you see in the crashlytics backend that the exception is reported from the old version or the new version? (Although I am not quite sure how does crashlytics get the version information. If it gets the information based on the apk installed on the system, then it will always reports the new version i think?)

[mvojtkovszky]

@beeender Looking through our crash logs, current pid value is not always less than opened one, for example

Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 4543. Current pid is 22644. in io_realm_internal_SharedGroup.cpp line 113

But that should not indicate that current process id is either "newer" or "older". Although PIDs are allocated on a sequential basis, when max value is reached, allocations start from 0 again.

Regarding Crashlytics reporting: as it is in our case initialised inside Application class which is started by a process under the same package, it will report a crash from whatever process causes it. Since we always see crash results matching the newer versions, I'd guess the new process throws the exception.

[beeender]

4543 is a quite small number for a pid, unless it is just powered on, i'd rather to assume it is "older" than 22644. But anyway, it is all guessing ...

Do you have any logic to restart a service in the apk? I tried something like START_STICKY, but cannot reproduce it :(

[mvojtkovszky]

The only time the app starts a service is when I bind IInAppBillingService to Activity.

But in any case, creating another process during runtime should crash the app in our situation anyway (since Realm instance is obtained in application class), which never happens. So that shouldn't prove anything.

Regarding your original hypothesis, did you manage to reproduce the "not-killed process on update" before? If so, it seems very plausible.

[beeender]

We tried a lot, but never reproduce "not-killed process on update" :(

But from the crash log, there are two processes for a same app are running in the system which should not ever happen ... We tried a lot but cannot reproduce it on any of our devices ...

[mvojtkovszky]

Is a following workaround currently possible for our situation: If there is a process that holds an open instance of Realm, is there a way to detect that and close that instance from second process that is about to obtain a new instance?

If not, solution to this is still pretty straightforward: Never obtain an instance of Realm in Application class, because that will keep it open for as long as that process is alive. Instead, always obtain instance when needed and close when done, without exceptions.

[beeender]

@biftekman Sorry for the late reply

If there is a process that holds an open instance of Realm, is there a way to detect that and close that instance from second process that is about to obtain a new instance?

The problem is it is really hard to detect the other process is holding an open instance for good or not.

Never obtain an instance of Realm in Application class, because that will keep it open for as long as that process is alive. Instead, always obtain instance when needed and close when done, without exceptions.

Yep, I think this could make the situation better. But one concern is if the system is still not doing the right thing, eg.: running into the old process's service which tries to open a Realm there, it still could happen :(

[beeender]

@biftekman Do you use firebase or any other libs could start a background process?

[mvojtkovszky]

@beeender no.

For the next update I'm working on a workaround, trying to catch a process about to be destroyed and make sure main Realm instance is closed before it does. Will let you know if it'll have any impact.

Also, if it would help, should I send you the crash reports? With about 5000 crashes in the last two weeks, might be some interesting data there :)

[beeender]

@biftekman Yes, please! also a apk would be helpful. You can send it to [email protected] if you want to share it privately. Thanks a lot!

[fjzboy]

01-05 13:16:16.556 17572-17572/? E/AndroidRuntime: FATAL EXCEPTION: main io.realm.exceptions.RealmError: Unrecoverable error. /data/data/com.xxx.xxxxx/files/caipu.realm: Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 16761. Current pid is 17572. in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 92 at io.realm.internal.SharedRealm.nativeGetSharedRealm(Native Method) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:205) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:182) at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:124) at io.realm.Realm.getDefaultInstance(Realm.java:209)

when I killed my app from TaskManager, then launch it again, this crash happened.

[kneth]

Which version of Realm do you use?

[fjzboy]

Hi, I use 2.2.0 version, and I find that this issue happened only on some of device(coolpad), and worked well on samsung s7.

[kneth]

@fjzboy Which model of coolpad? The reason I ask is that I wish to see if is reproducible on that Android version in general or if it's a device specific thing.

Can you try to upgrade to 2.2.2?

[dimmduh]

Same issue :( samsung s6 E/REALM_JNI: jni: ThrowingException 5, /data/data/com.offer4app.demo/files/default.realm: Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 3359. Current pid is 3303. in /home/cc/repo/realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 87, . 01-26 16:51:09.881 3303-3540/com.offer4app.demo E/REALM_JNI: Exception has been throw: Unrecoverable error. /data/data/com.offer4app.demo/files/default.realm: Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 3359. Current pid is 3303. in /home/cc/repo/realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 87 01-26 16:51:09.881 3303-3540/com.offer4app.demo E/REALM_JAVA: io.realm.exceptions.RealmError: Unrecoverable error. /data/data/com.offer4app.demo/files/default.realm: Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 3359. Current pid is 3303. in /home/cc/repo/realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 87 at io.realm.internal.SharedRealm.nativeGetSharedRealm(Native Method) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:208) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:186) at io.realm.RealmQuery$2.call(RealmQuery.java:1699) at io.realm.RealmQuery$2.call(RealmQuery.java:1692) at io.realm.internal.async.BgPriorityCallable.call(BgPriorityCallable.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:237) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) at java.lang.Thread.run(Thread.java:818)

`

01-26 16:51:09.931 3303-3303/com.offer4app.demo D/AndroidRuntime: Shutting down VM 01-26 16:51:09.931 3303-3303/com.offer4app.demo E/AndroidRuntime: FATAL EXCEPTION: main Process: com.offer4app.demo, PID: 3303 java.lang.Error: io.realm.exceptions.RealmError: Unrecoverable error. /data/data/com.offer4app.demo/files/default.realm: Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 3359. Current pid is 3303. in /home/cc/repo/realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 87 at io.realm.AndroidNotifier.throwBackgroundException(AndroidNotifier.java:138) at io.realm.RealmQuery.closeSharedRealmAndSendEventToNotifier(RealmQuery.java:2201) at io.realm.RealmQuery.access$000(RealmQuery.java:62) at io.realm.RealmQuery$2.call(RealmQuery.java:1722) at io.realm.RealmQuery$2.call(RealmQuery.java:1692) at io.realm.internal.async.BgPriorityCallable.call(BgPriorityCallable.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:237) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) at java.lang.Thread.run(Thread.java:818) Caused by: io.realm.exceptions.RealmError: Unrecoverable error. /data/data/com.offer4app.demo/files/default.realm: Encrypted interprocess sharing is currently unsupported.SharedGroup has been opened by pid: 3359. Current pid is 3303. in /home/cc/repo/realm/realm-java-release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 87 at io.realm.internal.SharedRealm.nativeGetSharedRealm(Native Method) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:208) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:186) at io.realm.RealmQuery$2.call(RealmQuery.java:1699) at io.realm.RealmQuery$2.call(RealmQuery.java:1692)  at io.realm.internal.async.BgPriorityCallable.call(BgPriorityCallable.java:36)  at java.util.concurrent.FutureTask.run(FutureTask.java:237)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)  at java.lang.Thread.run(Thread.java:818)  01-26 16:51:10.941 3303-3572/com.offer4app.demo I/System.out: (HTTPLog)-Static: isSBSettingEnabled false 01-26 16:51:10.941 3303-3572/com.offer4app.demo I/System.out: (HTTPLog)-Static: isSBSettingEnabled false 01-26 16:51:13.931 3303-3430/com.offer4app.demo I/System.out: (HTTPLog)-Static: isSBSettingEnabled false 01-26 16:51:13.931 3303-3430/com.offer4app.demo I/System.out: (HTTPLog)-Static: isSBSettingEnabled false 01-26 16:51:21.071 3303-3312/com.offer4app.demo E/System: Uncaught exception thrown by finalizer 01-26 16:51:21.071 3303-3312/com.offer4app.demo E/System: java.lang.NullPointerException: Null reference used for synchronization (monitor-enter) at io.realm.internal.SharedRealm.finalize(SharedRealm.java:348) at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:217) at java.lang.Daemons$FinalizerDaemon.run(Daemons.java:200) at java.lang.Thread.run(Thread.java:818) `

UPD. I disabled appmetrica sdk and it start working

[Zhuinden]

That's interesting, SharedRealm.finalize() got an NPE

[dimmduh]

@Zhuinden I used 2.2.2. I just updated to 2.3.0 and got same issue. I reproduced issue, and I guess it's not because of realm (but that's not good that's it crashes the app).

1. I debug connection on phone via proxy to fiddler
2. I use appmetrica sdk, when app start, appmetrica init and trying to connect, but cannot because fiddler change ssl sert and appmetrica server does not accept connection.
3. I close app and launch again => and got this issue I guess because appmetrica does not let app finish and realm is still occupied when I restart app. I guess it can happen not only with appmetrica but any sdk that can occupy app and not let it finish. I hope it's clear :)

[Zhuinden]

@dimmduh this is a breakthrough @cmelchior @kneth

[dimmduh]

Ohh... my mistake ... I put all my init code (realm, configs, etc) in MyApplication onCreate method, but the problem was I did not know appmetrica sdk works in details. Appmetrica sdk creates multiple application instances (to send crash logs even main app was crashed) that's why realm init was called twice.

[mvojtkovszky]

I've been experiencing this issue for the last 4 months or so :) However, ever since I moved realm init out of application's onCreate method, number of crashes dropped to insignificant numbers. Yet they still appear.

[stargt]

Fundamentally, This problem only can be solved by Encrypted interprocess sharing support. I am seriously expecting it to be fixed by an official update ASAP.