Mockk lost support for parallel test execution in version 1.13.7

[bay73]

It seems the version 1.13.7 has lost support for parallel execution. I would blame this change: https://github.com/mockk/mockk/pull/1099. clearAllMocks is known to not be thread safe and now it is called after each test. In previous version the issue with clearAllMocks could have been avoided by putting tests which use this function out of parallel execution scope. Now the issue affects all tests and parallel execution seems completely impossible.

• [X] I am running the latest version
• [X] I checked the documentation and found no answer
• [X] I checked to make sure that this issue has not already been filed

Expected Behavior

When using systemProperty("junit.jupiter.execution.parallel.enabled", "true") in gradle setting for test task I would expect test results to be the same as for disabled parallel execution.

Current Behavior

Many tests fail with io.mockk.MockKException: no answer found for <some mock> among the configured answers

Steps to Reproduce

Put systemProperty("junit.jupiter.execution.parallel.enabled", "true") into your build.gradle.kts file

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

• MockK version: 1.13.7
• Kotlin version: 1.9.0
• JUnit version: 5.6.2
• Type of test: unit test

[Evenprime]

Running into the same issue, when using the MockKExtension for unit-testing with latest mockK version, and the following settings enabled on a rather large project with over 1000 tests:

systemProperty("junit.jupiter.execution.parallel.enabled", "true")
systemProperty("junit.jupiter.execution.parallel.mode.classes.default", "concurrent")

So running classes in parallel, but tests within each class sequentially. In case it makes a difference for reproducing/tracking down the issue.

[Raibaz]

Ah, you're right, calling clearAllMocks on every test run is definitely going to break parallel execution.

Sorry about this.

Would it be helpful if we called clearAllMocks on every test run by default but added a configuration flag to disable it, so that you can prevent it from being called when running tests in parallel?

[Evenprime]

That would help me in my case already a lot.

[lifey]

@Raibaz Hi is there a plan to make clearAllMocks configurable ? Can I help somehow ? We cannot upgrade mockk as this break our parallelism

[Evenprime]

We ended up writing a custom test extension class and using that instead of the official MockKExtension implementation:

import io.mockk.MockKAnnotations
import io.mockk.unmockkAll
import org.junit.jupiter.api.extension.AfterAllCallback
import org.junit.jupiter.api.extension.ExtensionContext
import org.junit.jupiter.api.extension.TestInstancePostProcessor

class CustomMockKExtension : TestInstancePostProcessor, AfterAllCallback {
    override fun postProcessTestInstance(
        testInstance: Any,
        context: ExtensionContext,
    ) {
        MockKAnnotations.init(testInstance)
    }

    override fun afterAll(context: ExtensionContext) {
        unmockkAll()
    }
}

// Used simply as:

@ExtendWith(CustomMockKExtension::class)
class MyTestClass {
   // test class using @MockK, @InjectMockKs etc. as usual
}

That works well for us, as we don't use MockKs advanced features like static mocks, which would anyway prevent parallel tests. Depending on your usecases, you may need more logic from the official MockKExtension class copied in your own custom implementation.

[huisam]

Hello @Raibaz Does Mockk team has a plan to support a extension for parallel test? I cannot upgrade mockk version either as this issue

If not, I have to write the custom extension to maintain the parallel execution.

[Raibaz]

I took a look at this and the fix was easier than I expected.

I'm going to add a requireParallelTesting configuration setting to disable calling clearAllMocks after each test execution.

[vilikin]

Hello @Raibaz,

Is there maybe a bug still in the code that you added to fix the issue? It looks to me like clearAllMocks is called only when parallel testing is required, even though it should be the opposite - right? Or have I misunderstood something badly?

Also, regardless of how requireParallelTests is configured, it does not currently matter as if keepMocks is set to false (like it is by default), then that already forces clearAllMocks to be called.

[Evenprime]

vilikin is correct, the current implementation of that flag is broken and doesn't actually enable us to skip the call to clearAllMocks the way it was intended.

I attached a pull request with what I'd consider to be the proper logic for this flag.

[Raibaz]

Ah, you're right, thanks for looking into this and sending #1238!

I'll merge it as soon as the builds pass