verify wasNot Called leads to false positive

[manuNL]

Prerequisites

• [X] I am running the latest version
• [X] I checked the documentation and found no answer
• [X] I checked to make sure that this issue has not already been filed

Expected Behavior

wasNot in verify blocks should not lead to false positives, meaning tests are passing that actually should not pass. Even better would be that wasNot is not allowed on fields like functions and variables, but only on objects and otherwise does not even compile.

Current Behavior

I understood that wasNot Called should not be used on functions or other fields, only on whole object mocks. But recently a colleague used it on a function and we expected the test to fail. But the test actually passed and stated the function (or object) was not called, even it was. So we dug a bit deeper and found following behaviour:

verify { yourMock.functionWithRegularReturnType() wasNot Called }
verify { yourMock.functionWithRxJavaReturnType() wasNot Called }

A function with a regular return type will lead to a test crash like "can't find stub" or "can't find stub kotlin.". A function with a RxJava object, e.g. an observable, does not crash but the test passes (might be even with other 3rd party return objects as well).

Context

• MockK version: 1.13.1
• Kotlin version: 1.9.0
• JUnit version: 4.13.2
• Type of test: unit test

Stack trace

// -----------------------[ YOUR STACK STARTS HERE ] -----------------------

can't find stub 
io.mockk.MockKException: can't find stub 
	at app//io.mockk.impl.stub.StubRepository.stubFor(StubRepository.kt:16)
	at app//io.mockk.impl.recording.states.VerifyingState.checkWasNotCalled(VerifyingState.kt:79)
	at app//io.mockk.impl.recording.states.VerifyingState.recordingDone(VerifyingState.kt:45)
	at app//io.mockk.impl.recording.CommonCallRecorder.done(CommonCallRecorder.kt:47)
	at app//io.mockk.impl.eval.RecordedBlockEvaluator.record(RecordedBlockEvaluator.kt:64)
	at app//io.mockk.impl.eval.VerifyBlockEvaluator.verify(VerifyBlockEvaluator.kt:30)
	at app//io.mockk.MockKDsl.internalVerify(API.kt:119)
	at app//io.mockk.MockKKt.verify(MockK.kt:149)
	at app//io.mockk.MockKKt.verify$default(MockK.kt:140)
	at app//com.somepackage.ClassUnderTestTest.this test will crash with can't find stub kotlin String(ClassUnderTestTest.kt:53)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at [email protected]/java.lang.reflect.Method.invoke(Unknown Source)
	at app//org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at app//org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at app//org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at app//org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at app//org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at app//org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at app//org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at app//org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at app//org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at app//org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at app//org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at app//org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at app//org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at app//org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at app//org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at app//org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:40)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:60)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at [email protected]/java.lang.reflect.Method.invoke(Unknown Source)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at jdk.proxy1/jdk.proxy1.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
// -----------------------[ YOUR STACK TRACE ENDS HERE ] -----------------------

Minimal reproducible code (the gist of this issue)

import io.mockk.Called
import io.mockk.MockKAnnotations
import io.mockk.Runs
import io.mockk.every
import io.mockk.impl.annotations.MockK
import io.mockk.just
import io.mockk.verify
import io.reactivex.rxjava3.core.Observable
import org.junit.Before
import org.junit.Test

class ClassUnderTestTest {

    @MockK
    private lateinit var anotherTestClass: AnotherTestClass

    private lateinit var sut: ClassUnderTest

    @Before
    fun setUp() {
        MockKAnnotations.init(this)

        sut = ClassUnderTest(anotherTestClass)
    }

    @Test
    fun `this test will pass even getObservable was called`() {
        // Given
        every { anotherTestClass.getObservable() } returns Observable.never()
        every { anotherTestClass.getString() } returns ""
        every { anotherTestClass.getNothing() } just Runs

        // When
        sut.doSomeStuff()

        // Then
        verify { anotherTestClass.getObservable() wasNot Called }
    }

    @Test
    fun `this test will crash with can't find stub kotlin String`() {
        // Given
        every { anotherTestClass.getObservable() } returns Observable.never()
        every { anotherTestClass.getString() } returns ""
        every { anotherTestClass.getNothing() } just Runs

        // When
        sut.doSomeStuff()

        // Then
        verify { anotherTestClass.getString() wasNot Called }
    }

    @Test
    fun `this test will crash with can't find stub`() {
        // Given
        every { anotherTestClass.getObservable() } returns Observable.never()
        every { anotherTestClass.getString() } returns ""
        every { anotherTestClass.getNothing() } just Runs

        // When
        sut.doSomeStuff()

        // Then
        verify { anotherTestClass.getNothing() wasNot Called }
    }
}

class ClassUnderTest(
    private val anotherTestClass: AnotherTestClass
) {
    fun doSomeStuff() {
        anotherTestClass.getObservable()
        anotherTestClass.getString()
        anotherTestClass.getNothing()
    }
}

class AnotherTestClass {
    fun getObservable(): Observable<Boolean> = Observable.just(true)
    fun getString() = ""
    fun getNothing() = Unit
}

[Raibaz]

Yep this does indeed look like a bug to me.

A way to fix this would be to make sure wasNot was being called on a mocked object and throw an exception if it's called on a non-mocked object, so that verify { yourMock wasNot Called } works correctly and verify { yourMock.function() wasNot Called } would throw an exception.

[Gosunet]

I can try to propose a PR for what you said @Raibaz.

Checking if wasNot is being call on a mocked object if not i will throw an Exception.