koin icon indicating copy to clipboard operation
koin copied to clipboard

how to properly setup android UI tests?

Open or-dvir opened this issue 4 years ago • 10 comments

Hello. I can't seem to figure out how to properly setup my android UI tests. no metter what configuration i use i get some sort of koin related exception. here is my setup:

version used:

implementation "org.koin:koin-androidx-viewmodel:2.1.6"
testImplementation "org.koin:koin-test:2.1.6"
androidTestImplementation "org.koin:koin-test:2.1.6"

application class:

class MyApplication : Application() {

    companion object {
        val appModule = module {
            viewModel { FragmentUserListsViewModel(get()) }
            viewModel { FragmentNewEditListViewModel(get()) }
            viewModel { FragmentListItemsViewModel() }
            single<RepositoryUserLists> { RepositoryUserListsImpl() }
            factory<RepositoryListItems> { (listId: UUID) -> RepositoryListItemsImpl(listId) }
        }

        val testModule = module(override = true) {
            single<RepositoryUserLists> { RepositoryUserListsImplFake() }
            factory<RepositoryListItems> { (listId: UUID) -> RepositoryListItemsImplFake(listId) }
        }
    }

    override fun onCreate() {
        super.onCreate()

        startKoin {
            androidLogger()
            androidContext(this@MyApplication)
            modules(appModule)
        }
    }
}

test configuration 1:

    @get:Rule
    val koinTestRule = KoinTestRule.create {
        //so i can override the real repositories with fake ones
        modules(MyApplication.appModule, MyApplication.testModule)
    }

the first test always fails with java.lang.IllegalStateException: A KoinContext is already started. all other tests run fine and pass. its not a specific test that fails, but always the first one. here is the stack trace:

java.lang.IllegalStateException: A KoinContext is already started
at org.koin.core.context.KoinContextHandler.register(KoinContextHandler.kt:49)
at org.koin.core.context.ContextFunctionsKt.startKoin(ContextFunctions.kt:36)
at org.koin.core.context.ContextFunctionsKt.startKoin$default(ContextFunctions.kt:35)
at org.koin.test.KoinTestRule.starting(KoinTestRule.kt:25)
at org.junit.rules.TestWatcher.startingQuietly(TestWatcher.java:108)
at org.junit.rules.TestWatcher.access$000(TestWatcher.java:46)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:53)
at org.koin.test.mock.MockProviderRule$apply$1.evaluate(MockProviderRule.kt:13)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:392)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)

test configuration 2:

i thought that maybe MyApplication class already starts the koin context, and the KoinTestRule tries to start it again and thats why i get the exception. so i tried this:

    @Before
    fun beforeAndroidTest() {
        //so i can override the real repositories with fake ones
        loadKoinModules(MyApplication.testModule)
    }

    @After
    fun afterAndroidTest() {
        unloadKoinModules(MyApplication.testModule)
        stopKoin()
    }

now the first test always passes, and all other fail with No Koin Context configured. Please use startKoin or koinApplication DSL. here is the stack trace:

java.lang.IllegalStateException: No Koin Context configured. Please use startKoin or koinApplication DSL.
at org.koin.core.context.KoinContextHandler.getContext(KoinContextHandler.kt:29)
at org.koin.core.context.KoinContextHandler.get(KoinContextHandler.kt:35)
at org.koin.core.context.ContextFunctionsKt.unloadKoinModules(ContextFunctions.kt:67)
at com.hotmail.or_dvir.arislistkt2.BaseAndroidTest.afterAndroidTest(BaseAndroidTest.kt:60)
at java.lang.reflect.Method.invoke(Native Method)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at androidx.test.internal.runner.junit4.statement.RunAfters.evaluate(RunAfters.java:80)
at org.koin.test.mock.MockProviderRule$apply$1.evaluate(MockProviderRule.kt:13)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:392)
at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2189)

test configuration 3:

ok... so opposite problem, i must re-start it for all tests after the first one. i tried this:

@Before
    fun beforeAndroidTest() {
        //added this block
        startKoin {
            modules(MyApplication.appModule)
        }

        //so i can override the real repositories with fake ones
        loadKoinModules(MyApplication.testModule)
    }

    @After
    fun afterAndroidTest() {
        unloadKoinModules(MyApplication.testModule)
        stopKoin()
    }

but this is just back at square one with A KoinContext is already started

so whats the proper way to set up koin for android ui tests where i want to override some parts of a my module?

or-dvir avatar Jul 18 '20 16:07 or-dvir

So what is happening with your application is that when you start the application and it starts the given koin context. You also create the koin context inside the test in @Before section. So most likely the test start the koin context and then the application starts but fails because the test has already started the context.

So in your second try loading the koin modules just does not work because the context is not started because the application it self is not start and there for there is no koin context to loadKoinModules.

You need to think that the tests and the application are separate things. So you can't modify the applications dependencies outside the applications in this case from the tests.

But because in this case the application and the tests share the same virtualmachine and the same global space there are some possible solutions to handle this that pop in to my mind. Non of them are very nice solution.

I make the following assumption that you need to replace the repositories before the application starts.

  1. Base your module configuration on environment variables. Then you can have multiple configurations for starting up the application just pass the env variable and let the application load the expected configuration based on the environment variable. This would require you to write your stubbed / mocked repositories to the main/src space which you can exclude from the actual build by using proguard configuration.

  2. Have a global variable where you can set up module configuration and use it in side you Application module configuration.

  3. Have a separate Application that you start for the UI tests and there for you can start it from the gradle tests scope with the correct module configuration. If I remember correctly this will require you to some how modify the AndroidManifest to know what Application it's using and that can be difficult. This alternative is similar to the first one.

This is hypothetical but could work 4) instead of single scoped you could replace the whole viewmode and repository scoping with factory scope and then you could replace the repository modules in the test functions because the application has already started the context and when you load a view the viewmodel and repositories would be created from the replaced modules when the view loads.

Hope that these ideas help you to understand the actual problem.

This is actually a very interesting problem and not an easy way to solve. I would like to hear if you figure out how to do this. Would benefit the whole community.

niom avatar Aug 13 '20 12:08 niom

a few days ago i came up with something: i removed KoinTestRule', the startKoinblock, andstopKoin- leaving onlyloadKoinModulesandunloadKoinModules`

    @Before
    fun beforeAndroidTest() {
        loadKoinModules(MyApplication.testModule)
    }
    @After
    fun afterAndroidTest() {
        unloadKoinModules(MyApplication.testModule)
    }

this seems to work, although im not sure 100% why. as i am new to testing, i can't be sure but i have a theory: i thought that the testing process (meaning the process which the operating system runs) is restarted every time for each test (similar to @Before and @After), but i guess that while the app itself is restarted, the process is kept alive. this means that the Application class is started only once (same as the process), and this would explain the exceptions:

  • test configuration 1 and 3: Application class starts -> koinTestRule/startKoin is trying to start a koinContext but it is already started from Application -> "koin context already started" exception.

  • test configuration 2: Application class starts -> first test is run successfully -> stopKoin is called in @After -> next test tries to run but koinContext is now destroyed -> "no koin context configured" exception.

by only using loadKoinModules and unloadKoinModules, the koinContext is only started once by the Application class, and it destroyed by the operating system when the entire test run finishes.

please note again that i am not sure this is actually how it works - this is just a theory that seems to make sense. if someone who is smarter than me can confirm this , that would be great :) (denying it and providing another explanation is also good!)

or-dvir avatar Aug 13 '20 13:08 or-dvir

So far ended up using something similar @or-dvir , thanks.

oradkovsky avatar Aug 20 '20 21:08 oradkovsky

Found a better way imo: https://stackoverflow.com/questions/63519198/proper-instrumentation-test-with-koin/63524040#63524040

oradkovsky avatar Aug 21 '20 13:08 oradkovsky

better? it looks exactly the same... only difference is your using mocks and im using fakes.

anyways glad it works for you

or-dvir avatar Aug 21 '20 15:08 or-dvir

Doesn't work for me.

In test without mocked module which goes after test with mocked module there is an exception org.koin.core.error.NoBeanDefFoundException: No definition found for class:...

I tried both: with parameter override = true and override = false.

So, I have same question. What is the proper way to mock something only for one test in android UI tests. Is there any example or best practice?

sgulyaev avatar Nov 17 '20 16:11 sgulyaev

It also doesn't work for me with 2.2.1

    implementation( "org.koin:koin-androidx-viewmodel:2.2.1")
    androidTestImplementation( "org.koin:koin-test:2.2.1")

I'm starting a new single-activity project using MVVM. Our team heavily uses TDD, so I'm trying to establish the right way to test Fragments that interract with ViewModels. Ideally, we'd be able to inject mock viewmodels before each instrumented test and assert on interractions. There are a few different approaches out there.

It would be really helpful to provide a recommended way to do this in the documentation, if possible.

tscott0 avatar Dec 23 '20 17:12 tscott0

Since my normal Application.kt starts Koin and does all kinds of inits, I created a custom test runner that starts an empty Application. That way I can start Koin in the test's @Before and only load the modules that the SUT is concerned with.

class MyTestRunner : AndroidJUnitRunner() {
    override fun newApplication(cl: ClassLoader?, className: String?, context: Context?): Application {
        return super.newApplication(cl, TestApplication::class.java.name, context)
    }
}
@SuppressLint("Registered")
class TestApplication : SplitCompatApplication() {
    override fun onCreate() {
        super.onCreate()
        // don't start koin
        Timber.plant(Timber.DebugTree())
    }
}

Then use the runner in module's build.gradle:

defaultConfig {
        testInstrumentationRunner "com.my.package.MyTestRunner"
}

With that setup, you should be able to start your test Koin instance in @Before with whatever mocks you wish.

HOWEVER, I faced a problem with testing Dynamic Feature Module's "initial" Activity/Fragment, since I inject the feature module's modules in that class's onCreate.

To circumvent loading DFM's modules in my test, I created a little extension to use in tests:

private const val TESTING = "TESTING"

var Koin.isTesting: Boolean
    get() = getProperty(TESTING, false)
    set(value) {
        setProperty(TESTING, value)
    }

Usage in test:

class IntroActivityTest {

    @Before
    fun setUp() {
        startKoin {
            koin.isTesting = true // <-- set testing true
            modules(
                // your mock modules
            )
        }
    }

    @After
    fun tearDown() {
        stopKoin()
    }

    @Test
    fun foo() {
        val scenario = ActivityScenario.launch(IntroActivity::class.java)
        // your test logic
        scenario.close() // important!
    }
}

Usage in DFM Activity / Fragment

private val loadModules by lazy {
    loadKoinModules(introModules)
}

private fun injectFeatures() = loadModules

class IntroActivity : BaseActivity<ActivityIntroBinding>() {

    private val mIntroViewModel: IntroViewModel by viewModel()

    override fun onCreate(savedInstanceState: Bundle?) {
        if (!getKoin().isTesting) {
            injectFeatures()
        }
        super.onCreate(savedInstanceState)
        _binding = ActivityIntroBinding.inflate(layoutInflater)
        setContentView(binding.root)
    }
}

Remember to call scenario.close() at the end of your test, or you might run into java.lang.IllegalStateException: KoinApplication has not been started. I tracked down the cause of the exception and it was due to my Activity's lifecycleScope.launchWhenResumed{} trying to get a Koin dependency AFTER @After-block had been run!

TL;DR: The activity you launch with ActivityScenario.launch() won't close before @After unless you call scenario.close()!!!

vpuonti avatar Feb 04 '21 14:02 vpuonti

My test with replace Room DB instance with inMemoryDatabaseBuilder


    @KoinReflectAPI
    fun test() = module {
        single<AppDatabase> {
            Room
                .inMemoryDatabaseBuilder(get(), AppDatabase::class.java)
                .fallbackToDestructiveMigration()
                .build()
        }
        single<AppDataRepository> {
            AppDataRepositoryImpl(
                scope = get(),
                ioDispatcher = Dispatchers.IO,
                database = get()
            )
        }
    }


@KoinReflectAPI
@RunWith(AndroidJUnit4::class)
class AppDataRepoTest : KoinTest {
    @Before
    fun beforeAndroidTest() {
        loadKoinModules(TestKoinModule.test())
    }

    @After
    fun afterAndroidTest() {
        unloadKoinModules(TestKoinModule.test())
    }

    @Test
    fun repoTest() {
        runBlocking {
            val dataRepository = get<AppDataRepository>()
          
           
        }
    }
}

a-reznic avatar Dec 27 '21 14:12 a-reznic

Any updates on this?

At the moment I'm using a TestRule that was inspired by this blog post: https://medium.com/stepstone-tech/better-tests-with-androidxs-activityscenario-in-kotlin-part-1-6a6376b713ea

LazyKoinActivityScenarioRule.kt:

import android.app.Activity
import android.content.Intent
import androidx.test.core.app.ActivityScenario
import org.junit.rules.ExternalResource
import org.koin.core.component.KoinComponent
import org.koin.core.context.loadKoinModules
import org.koin.core.context.unloadKoinModules
import org.koin.core.module.Module

/**
 * Allows to launch an activity lazy so that setup can be done before the activity is opened
 * Copied from https://medium.com/stepstone-tech/better-tests-with-androidxs-activityscenario-in-kotlin-part-1-6a6376b713ea
 * and changed to run with Koin
 */
class LazyKoinActivityScenarioRule<A : Activity> : ExternalResource, KoinComponent {

    constructor(
        launchActivity: Boolean,
        koinOverrideModule: Module? = null,
        startActivityIntentSupplier: () -> Intent,
    ) {
        this.launchActivity = launchActivity
        this.overrideModule = koinOverrideModule
        scenarioSupplier = { ActivityScenario.launch(startActivityIntentSupplier()) }
    }

    constructor(
        launchActivity: Boolean,
        startActivityIntent: Intent,
        koinOverrideModule: Module? = null,
    ) {
        this.launchActivity = launchActivity
        this.overrideModule = koinOverrideModule
        scenarioSupplier = { ActivityScenario.launch(startActivityIntent) }
    }

    constructor(
        launchActivity: Boolean,
        startActivityClass: Class<A>,
        koinOverrideModule: Module? = null,
    ) {
        this.launchActivity = launchActivity
        this.overrideModule = koinOverrideModule
        scenarioSupplier = { ActivityScenario.launch(startActivityClass) }
    }

    private var launchActivity: Boolean

    private var scenarioSupplier: () -> ActivityScenario<A>

    private var scenario: ActivityScenario<A>? = null

    private var scenarioLaunched: Boolean = false

    private var overrideModule: Module? = null

    override fun before() {
        overrideModule?.let { loadKoinModules(it) }
        if (launchActivity) {
            launch()
        }
    }

    override fun after() {
        scenario?.close()
        overrideModule?.let {
            unloadKoinModules(it)
            loadKoinModules(koinAppModules)
        }
    }

    fun launch(newIntent: Intent? = null) {
        if (scenarioLaunched) throw IllegalStateException("Scenario has already been launched!")

        newIntent?.let { scenarioSupplier = { ActivityScenario.launch(it) } }

        scenario = scenarioSupplier()
        scenarioLaunched = true
    }

    fun getScenario(): ActivityScenario<A> = checkNotNull(scenario)
}

inline fun <reified A : Activity> lazyActivityScenarioRule(
    launchActivity: Boolean = true,
    koinOverrideModule: Module? = null,
    noinline intentSupplier: () -> Intent,
): LazyKoinActivityScenarioRule<A> =
    LazyKoinActivityScenarioRule(launchActivity, koinOverrideModule, intentSupplier)

inline fun <reified A : Activity> lazyActivityScenarioRule(
    launchActivity: Boolean = true,
    intent: Intent? = null,
    koinOverrideModule: Module? = null,
): LazyKoinActivityScenarioRule<A> = if (intent == null) {
    LazyKoinActivityScenarioRule(launchActivity, A::class.java, koinOverrideModule)
} else {
    LazyKoinActivityScenarioRule(launchActivity, intent, koinOverrideModule)
}

Hint: koinAppModules is defined globally and also used in my Application's startKoin-Block.

Usage example:

import io.mockk.every
import io.mockk.mockk
import io.mockk.verify
import org.junit.Rule
import org.junit.Test
import org.koin.dsl.module

class EnhancedMainActivityTest {

    @get:Rule
    val mainActivityRule = lazyActivityScenarioRule<MainActivity>(
        koinOverrideModule = module { single { viewModel } }
    )

    private val viewModel = mockk<KoinViewModel>(relaxed = true) {
        every { getItems() } returns listOf()
    }

    @Test
    fun test() {
        verify { viewModel.getItems() }
    }
}

sarn0ld avatar Apr 25 '22 13:04 sarn0ld

I'm new to Koin, but just ran into a similar problem when trying to use it in Espresso tests, and I think the problem with your first scenario is that your Application class is starting koin (as if it was running the app normally), then you're starting koin again. I didn't write tests in the same way, but I got around the same issue (of starting koin twice) by simply calling stopKoin before starting it. This could be cleaner by overriding the application class via a custom test runner though.

To mock my injected dependencies though, I just did the following as a quick and easy solution that had minimal effect on the prod/main code:

class App : Application() {
    override fun onCreate() {
        super.onCreate()
        startKoin {
            androidContext(this@App)
            modules(ModulesProvider.modules)
        }
    }
}

@VisibleForTesting
object ModulesProvider {
    val modules = listOf(networkingModule, dashboardModule)
}
class KoinModuleRule(private val modules: List<Module> = emptyList()) : TestWatcher() {

    override fun starting(description: Description) {
        super.starting(description)
        // Since we're not override the application class, we're starting koin on startup of app so need to stop that instance
        stopKoin()
        startKoin {
            modules(ModulesProvider.modules.toMutableList().apply { addAll(modules) })
        }
    }

    override fun finished(description: Description) {
        super.finished(description)
        stopKoin()
    }
}
@RunWith(AndroidJUnit4::class)
class DashboardFragmentTest {

    @get:Rule
    val koinModuleRule = KoinModuleRule(listOf(mockDashboardModule))

    @Test
    fun testDashboardActivity() {
        launchFragmentInContainer<DashboardFragment>()

        Espresso.onView(ViewMatchers.withText("Dashboard 12"))
            .check(ViewAssertions.matches(ViewMatchers.isDisplayed()))
    }
}

private val mockDashboardRepo: DashboardRepo = mockk() {
    every { printRandom() } returns "12"
}
val mockDashboardModule = module {
    single { mockDashboardRepo }
}

This is all super basic code however as I'm just learning Koin in a multi module empty project, so I'm not sure yet how it will stack up with more advanced usage (e.g., scoping etc.).

danieljonker avatar Aug 19 '22 15:08 danieljonker

Please follow updated doc introduced in https://github.com/InsertKoinIO/koin/pull/1409

arnaudgiuliani avatar Aug 29 '22 14:08 arnaudgiuliani