Allow for switching between NiceMock and StrictMock at runtime

[cbandera]

trafficstars

Why do we need this feature?

In my unittests I follow the AAA pattern (Arrange, Act, Assert), also I like to use test fixtures for moving common arrange steps into the SetUp() method. When using StrictMocks, every call to the mock object must be registered as expected. This get's difficult or at least cumbersome if I need to perform mutliple steps during the arrange statement.

Example: Consider following example: Given a statemachine that acts on a mock object. I want to write dedicated tests for each state and strictly test, that the protocol within this state is adhered to (here I want StrictMock behaviour). But as my API of the statemachine is designed to prevent misusage, I need to switch through a couple of states first for reaching my to-be-tested state (here I want NiceMock behaviour).

Describe the proposal

Given the current implementation, the decision of whether a Mock is Nice or Strict is already a runtime decision. Only the user interface to it is designed to be a compiletime decision. I propose to extend the interface by providing the following functions in gmock-spec-builders.h:

class Mock {
 public:
  ...
  // Converts the mock into a naggy mock (default)
  static void MakeNaggy(void* mock_obj);
  // Converts the mock into a nice mock
  static void MakeNice(void* mock_obj);
  // Converts the mock into a strict mock
  static void MakeStrict(void* mock_obj);
...

In the definition, these functions would then modify the static lookup map, the way the constructors of the Nice/StrictMock wrappers do.

Remarks I did try out several things, like making use of VerifyAndClearExpectations but it is explicitly stated in the documentation that it's undefined behaviour to set new expectations after this call. My current solution is the "Check Point" solution from the CookBook, but it's rather cumbersome to set up all "wildcard" expectations for every method of every mock.

Maybe there is a design decision that I am not aware of, which discourages the proposed feature or maybe there is a existing way of achieving the same. If so, I'd be happy to hear about it. If not, I'd be happy to draft a PR with the requested feature.

[CJ-Johnson]

Thank you for this feature request! This is not something we can prioritize, but it does seem compelling. In order to adopt this, we would need to devote time to understanding the API implications and going through a review process. It's been added to the hopper, but it probably will not happen any time soon, unfortunately.

[cbandera]

In the meanwhile I dug a little deeper and at the moment I think, this would technically be possible. But as you correctly stated, the implications for the API need to be considered. At the moment users have the guarantee of type safety for their mock class. If you can effectively convert a StrictMock to a NiceMock at runtime, this could lead to some unintended surprises. So maybe it's best to restrict this new API only to "classical" Mocks...

For anyone interested how I solved this issue with the current state, here are two options:

Option 1: Correct usage of API

The only way I found to implement my feature with the correct usage of the current api is to be using checkpoints. Check the link, it describes this strategy pretty well.

class MyMock : public MyInterface {
public:
   MOCK_METHOD(void, foo, (), (override));
   MOCK_METHOD(void, bar, (), (override));
   ...
};

class MyFixture : public testing::Test {
public:
  MockFunction<void()> endArrangeStatements; // Checkpoint for starting the test

  void SetUp() override {
    ExpectationSet allowed_arrange_calls;
    allowed_arrange_calls += EXPECT_CALL(mock, foo).Times(gt::AnyNumber());
    allowed_arrange_calls += EXPECT_CALL(mock, bar).Times(gt::AnyNumber());
    // Continue this for every mock and every mocked method ....

    EXPECT_CALL(endArrangeStatements, Call()).After(allowed_arrange_calls); // The expect statements from above are only valid until the MockFunction was called.
  }

protected:
   StrictMock<MyMock> mock{};
};

TEST_F(MyFixture, DummyTest){
   // Set further expectations on mock

   // Do anything with the mock to get it or other objects into the desired state
  this->endArrangeStatements.Call();

  // Start the actual test sequence
}

Option 2: The hacky but slim approach

DISCLAIMER: This solution uses knowledge of the implementation details and as such is subject to breaking anytime if you live from HEAD. Keep this in mind....

While the first approach worked, it just felt wrong and cumbersome to add a EXPECT_CALL(anything) statement for every mocked method. What I need is really a "catch-all" statement. I use the fact that both NiceMock and StrictMock inherit from the Mock object itself, and that they don't add any new members or functions and hence don't change the memory layout.

class MyMock : public MyInterface {
public:
   // Implement copy and assignment operator for your mock
   MyMock (const MyMock &other);
   MyMock &operator=(const MyMock &other);

   MOCK_METHOD(void, foo, (), (override));
   MOCK_METHOD(void, bar, (), (override));
   ...
};

class MyFixture : public testing::Test {
public:
  void SetUp() override {
    mock = NiceMock<MyMock>(); // Initialize instance with a NiceMock obj
  }

  void endArrangeStatements(){
    mock = StrictMock<MyMock>(mock); // Replace instance with a StrictMock copy of it
  }

protected:
   MyMock mock{};
};

TEST_F(MyFixture, DummyTest){
   // Do anything with the mock to get it or other objects into the desired state
  this->endArrangeStatements();

  // Set further expectations on mock

  // Start the actual test sequence
}

So in the SetUp method, I initialize the object with a NiceMock. Since I can't hold a NiceMock itself, the temporarily created NiceMock will be implicitly cast to a MockObject again, but the important part is: Within the constructor of the NiceMock, the niceness is already registered in the global gmock registry. In the endArrangeStatements functions, I then replace the current object with a StrictMock copy of it. Again, the constructor set's the global flag. The copyconstructor takes over any real members the mock might have. The assignment operator implicitly casts the object back to a real Mock object. With this approach, we are really exchanging mock objects here. Anyone holding a reference to it won't mind. But the important thing is that you need to set your EXPECT_CALL statements after the call to endArrangeStatements so that they are set on the correct object. Please note also that your Mock object might require you to define your copy and assignment operator explicitly.

[PieterBekaert]

I came across the same scenario where you only want strictness after some initialization. In particular, one wants to ignore calls that happened up to the point where the actual test start. So maybe a solution that has less impact could be to stick to the static distinction between nice and strict, but to offer a way to discard all calls to the strict mock right before you start with the actual test?

[mortenfc]

Bump