realm-kotlin icon indicating copy to clipboard operation
realm-kotlin copied to clipboard

Add guidance around unit testing

Open cmelchior opened this issue 3 years ago • 16 comments

Similar to https://www.mongodb.com/docs/realm/sdk/java/advanced-guides/testing/ we should provide guidance for testing Realm Kotlin in combination with common test frameworks, especially Roboletric.

Since Realm Kotlin has support for running directly on the JVM, this should be much easier to achieve, but we should make sure we have tested this.

Maybe we should add a project demonstrating both Android and Multiplatform tests to realm-kotlin-samples

cmelchior avatar Aug 02 '22 12:08 cmelchior

Any updates on this? I am struggling a bit on how to make unit testing work when migrating from the java SDK to the Kotlin sdk. In particular testing view model functions that use realm.write.

sipersso avatar Mar 17 '23 13:03 sipersso

If anyone else is struggling with this after migrating to the kotlin SDK. For testability it is a huge issue that Realm.write uses it's own write dispatcher that is not possible to override. This is a major pain when writing tests. You can't use queries to assert code that writes to the database since the write operation is async (and haven't completed writing yet). You may also run into deadlocks if your code is using realm.write multiple times (or if running tests in parallel).

In my case I made a workaround. The workaround assumes that your tests are running in memory realms and production code is running file based realms. The idea is that the in memory realms use a blocking write to make it easier to test and avoid deadlocks, while the production codes regular realm.write.

suspend fun <R> Realm.writeAsyncInProdAndBlockingForTests(block: MutableRealm.() -> R):R{
    return if(configuration.inMemory) {
        writeBlocking(block)
    } else {
        write(block)
    }
}

By using this approach I was finally able to migrate all tests over to Kotlin, but it was very very tricky to get this to work. Hope this is helping others that run into the same issue as I did.

sipersso avatar Apr 20 '23 10:04 sipersso

@sipersso

We actually do have a RealmConfiguration.Builder.writeDispatcher() method but it is currently marked internal because we weren't sure if it was worth exposing it, but given your use case, we should probably reconsider that: https://github.com/realm/realm-kotlin/blob/main/packages/library-base/src/commonMain/kotlin/io/realm/kotlin/Configuration.kt#L299

You can already access it today if you use the @Suppress("invisible_reference", "invisible_member") annotation though. Something like:

@Suppress("invisible_reference", "invisible_member")
RealmConfiguration.Builder(schema)
  .writeDispatcher(getDispatcher)
  .build()

We haven't tested it for all edge cases though, so be aware if you start using it. The biggest requirement is that it must be a dispatcher backed by a single thread.

cmelchior avatar Apr 21 '23 08:04 cmelchior

@sipersso

We actually do have a RealmConfiguration.Builder.writeDispatcher() method but it is currently marked internal because we weren't sure if it was worth exposing it, but given your use case, we should probably reconsider that: https://github.com/realm/realm-kotlin/blob/main/packages/library-base/src/commonMain/kotlin/io/realm/kotlin/Configuration.kt#L299

You can already access it today if you use the @Suppress("invisible_reference", "invisible_member") annotation though. Something like:

@Suppress("invisible_reference", "invisible_member")
RealmConfiguration.Builder(schema)
  .writeDispatcher(getDispatcher)
  .build()

We haven't tested it for all edge cases though, so be aware if you start using it. The biggest requirement is that it must be a dispatcher backed by a single thread.

No wories, the workaround works well in the meantime. Not 100% sure what is meant by that the dispatcher must be backed by a single thread, but I'd expect to be able to use the UnconfinedTestDispatcher? Anyway, great that you are considering this case. I suspect a lot of people will be writing unit tests for code on Realm Kotlin ;)

sipersso avatar Apr 21 '23 08:04 sipersso

Hi folks, True, some guidance around unit testing would be helpful.

In the meantime, could you please advise -- does in-memory realm-kotlin support unit tests without mocking of the Realm? (on JVM 17, intel x64 CPU)

Simple opening of the realm (v. 1.18.0) throws:

java.lang.ExceptionInInitializerError
	at io.realm.kotlin.RealmConfiguration$Builder.<init>(RealmConfiguration.kt:55)
	at Test.setup(Test.kt:15)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	...
Caused by: java.lang.NullPointerException: {
        @Suppress("DEP…id.os.Build.CPU_ABI
    } must not be null
	at io.realm.kotlin.internal.platform.SystemUtilsAndroidKt.<clinit>(SystemUtilsAndroid.kt:13)

kzotin avatar May 04 '23 16:05 kzotin

I run into the same issue as @kzotin. We really need some guidance/examples. I had to comment out some integration tests now, after converting to Realm.

ZsoltBertalan avatar May 18 '23 14:05 ZsoltBertalan

Unit tests work with latest release without any surprises and Robolectrics, kudos for that! Guess issue can be closed

kzotin avatar Jul 14 '23 12:07 kzotin

Maybe the RoboElectric issue can be closed, but there is still no guidance around unit testing for the Kotlin lib.

sipersso avatar Jul 14 '23 12:07 sipersso

hi Returned to simple Unit test, and it fails again. Something to do with RealmLog

java.lang.ExceptionInInitializerError
	at io.realm.kotlin.log.RealmLog.addDefaultSystemLogger(RealmLog.kt:124)
	at io.realm.kotlin.log.RealmLog.<clinit>(RealmLog.kt:48)
	at io.realm.kotlin.Configuration$SharedBuilder.<init>(Configuration.kt:229)
	at io.realm.kotlin.RealmConfiguration$Builder.<init>(RealmConfiguration.kt:53)
	at com.ftband.shared.realm.RealmKtStorageTest.setup(RealmKtStorageTest.kt:20)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:112)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:60)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.NullPointerException: {
        @Suppress("DEP…id.os.Build.CPU_ABI
    } must not be null
	at io.realm.kotlin.internal.platform.SystemUtilsAndroidKt.<clinit>(SystemUtilsAndroid.kt:16)
	... 50 more

kzotin avatar Nov 23 '23 14:11 kzotin

@kzotin

Caused by: java.lang.NullPointerException: {
        @Suppress("DEP…id.os.Build.CPU_ABI
    } must not be null
	at io.realm.kotlin.internal.platform.SystemUtilsAndroidKt.<clinit>(SystemUtilsAndroid.kt:16)
	... 50 more

This error here would be thrown from earlier versions of Realm.

What happens if you upgrade to the latest 1.12.1? If that still fails, can you reproduce it in a sample project? It would be very helpful to figure out what is wrong.

cmelchior avatar Nov 23 '23 14:11 cmelchior

Reproduced on 1.12.0, and 1.12.1 not yet released. How could I address snapshot version?

kzotin avatar Nov 23 '23 15:11 kzotin

Ah sorry, testing on 1.12.0 is fine. Would it be possible to you to create a sample project we can take a look at? Because somehow your Gradle setup must be different than the ones we use.

cmelchior avatar Nov 23 '23 15:11 cmelchior

Simple junit test (JVM) used to work, now it doesn't

    class TestItem : RealmObject {
        @PrimaryKey
        var name: String = ""
    }
    
    @Test
    fun realmSimpleTest() {
        val config = RealmConfiguration.Builder(schema = setOf(TestItem::class))
            //.inMemory()
            .build()

    }

kzotin avatar Nov 23 '23 15:11 kzotin

oh, btw it stopped after switching from intel i7 to Macbook M2, shoult be it

kzotin avatar Nov 23 '23 15:11 kzotin

I tried to run your test inside https://github.com/realm/realm-kotlin-samples/tree/main/JVMConsole and bumped it to Kotlin 1.9.10 and Realm 1.12.0. Your test, both with and without inMemory works fine there.

You can try to replicate it yourself and if that also works on your machine it must be some problem in your current project setup. But maybe you can compare the two projects and find the difference.

cmelchior avatar Nov 23 '23 16:11 cmelchior

We had a version mismatch between gradle plugin (1.12) & library (1.11.1). After fixing everything works! Sorry for confusion

kzotin avatar Nov 24 '23 08:11 kzotin