Bug: Unexpected method calls are "forgotten"

[Groxx]

trafficstars

In a nutshell: this test passes, despite the expectations being incorrect:

https://go.dev/play/p/2r54p3TxXqr

package durable

import (
	"testing"
	"github.com/stretchr/testify/mock"
)

type mymock struct {
	mock.Mock
}

func (m *mymock) Run() error {
	args := m.Called()
	return args.Error(0)
}

func TestPanics(t *testing.T) {
	m := new(mymock)
	run := func() {
		defer func() { recover() } () // intentionally suppress panic

		m.On("Run").Return(nil).Once()

		m.Run()
		m.Run() // panics
		t.Fatal("Unreachable")
	}

	run()

	m.AssertExpectations(t) // passes
}

Granted, this isn't great code. But panic-capturing code is somewhat common in frameworks, and this can lead to tests passing despite clearly having incorrect mocks. The fact that AssertExpectations doesn't match behavior seems like a bug in testify.

---

So far as I can tell, this is caused by this code below, in mock/mock.go. Because the panic occurs before the call.totalCalls++ op (and possibly others!), no record of a panic-ing call exists except for the panic.

func (m *Mock) MethodCalled(methodName string, arguments ...interface{}) Arguments {
	m.mutex.Lock()
	found, call := m.findExpectedCall(methodName, arguments...)

	if found < 0 {
		// we have to fail here - because we don't know what to do
		// as the return arguments.  This is because:
		//
		//   a) this is a totally unexpected call to this method,
		//   b) the arguments are not what was expected, or
		//   c) the developer has forgotten to add an accompanying On...Return pair.

		closestFound, closestCall := m.findClosestCall(methodName, arguments...)
		m.mutex.Unlock()

		if closestFound {
			panic(fmt.Sprintf("\n\nmock: Unexpected Method Call\n-----------------------------\n\n%s\n\nThe closest call I have is: \n\n%s\n\n%s\n", callString(methodName, arguments, true), callString(methodName, closestCall.Arguments, true), diffArguments(closestCall.Arguments, arguments)))
		} else {
			panic(fmt.Sprintf("\nassert: mock: I don't know what to return because the method call was unexpected.\n\tEither do Mock.On(\"%s\").Return(...) first, or remove the %s() call.\n\tThis method was unexpected:\n\t\t%s\n\tat: %s", methodName, methodName, callString(methodName, arguments, true), assert.CallerInfo()))
		}
	}

	if call.Repeatability == 1 {
		call.Repeatability = -1
	} else if call.Repeatability > 1 {
		call.Repeatability--
	}
	call.totalCalls++
	// ...
}

I don't know what all would be involved in correcting this though. Ideally this should both panic (which generally aborts the test prematurely) and record the error for reporting in AssertExpectations in case it's suppressed.

[Groxx]

Oops. Looks like I'm on an older one, this might be resolved now. I'll dive through the code now / that's released and close this if so.

[Groxx]

Nope. It's much-improved by the addition of the testing.T injection (thank you @posener !), but non-test-injected ones still behave like this.

I mean, I'm going to upgrade and put testing.T in there immediately in my code, that's excellent. But there's quite a lot of older code that could still benefit from fixing this.

[devdinu]

@Groxx You could do t.Fail or t.Fatal inside the defer func which handles panic recovery to fail the test with error. If you're mentioning the defer as part of the app code, we should look into that.

AssertExpectations currently verifies whether all the expectations On("method").Return are met. If you're suggesting call to maintain state, we could but anyways the method call without expectation would panic, and hard to change existing AssertExpectations for this specific case.

Feel free to raise a PR for changes/refactoring.

[KrzysztofMadejski]

Why did you, library creators, decided to panic on "Unexpected Method Call" rather than fail a test in AssertExpectations or earlier if one could inject t? In my opinion flagging errors would have a lot cleaner behaviour.