HMock
HMock copied to clipboard
Consider supporting the log-and-verify approach
There are two different approaches to modern mocking. Libraries like gMock are plan-and-test, where you first declare an execution plan, then run the test and the execution plan is verified during and after the test run. Libraries like Mockito are log-and-verify, where the test runs first and then assertions are made about the logged call list.
Currently, HMock follows the plan-and-test approach. I think there are advantages to doing it this way, because the desired response to actions is usually tied to why those actions are expected in the first place. So the log-and-verify approach sometimes requires repeating yourself to declare a return value at the beginning and then verify the actual call at the end. There are two different languages for this (Mockito's when
versus verify
)
HMock can do both, though. It can keep a log of actions as the test runs, and also allow the user to verify that call log against another set of expectations after the test has completed. Verification can just be another ExpectContext
instance, so that the same primitives are available for verification as for advance expectation.
For example:
runMockT $ do
allowUnexpected $ ReadFile_ anything |-> "contents"
allowUnexpected $ WriteFile anything anything
copyFile "foo.txt" "bar.txt"
verify $ inSequence [
ReadFile "foo.txt",
WriteFile "bar.txt" "contents"
]
The idea is that the user can choose to be very permissive when setting up responses, but then run a more specific verification (maybe even several of them!) after the fact. If the verify
block has return values, it should fail at runtime. (We cannot compare, because we don't have Eq instances.)
The semantics here are unclear. A few options:
- Verify the full action sequence against the expect set passed to verify. In a complex test, though, it's unlikely that you want to verify the full action sequence each time.
- Verify that some subset of the action sequence matches. Aside from possible exponential blowup of matching, though, this leaves no way to say "this was never called", since
verify
will always match the subsequence that ignores that action. - Mockito's verification is per-method, and non-matching methods are ignored. The equivalent here would be to ignore unmentioned methods. This is the "uninteresting" trick again, but applied only within the scope of
verify
. It has some of the same compositionality problems, but they are mitigated by the fact that you can callverify
multiple times for independent expectations.
I'm deciding on option 3.