Setting side effects after creating expectactions (inside DOCTEST_SUBCASEs)

[syyyr]

Hi,

the application I'm testing blocks when it is run through it's exec() method (it's a Qt application). Because of this, I have to set up all of my expectations before I run the apllication. I rely heavily on doctest's SUBCASE function to reuse the common code for test cases. An example interface for my Application class looks like this.

class Interface {
public:
	virtual ~Interface() = default;
	virtual void init() = 0;
	virtual void someMethod() = 0;
	virtual void otherMethod() = 0;
};

class MyApp {
public:
	MyApp(Interface*);
	void exec();
};

and I implement the mock like this:

class MockInterface : public trompeloeil::mock_interface<Interface> {
	IMPLEMENT_MOCK0(init);
	IMPLEMENT_MOCK0(someMethod);
	IMPLEMENT_MOCK0(otherMethod);
};

Now, for example, let's say that after I call the exec() method of MyApp, it is supposed to:

• call the init() method first
• call someMethod() whenever the conditions for it apply (some side effect)
• call otherMethod() whenever the conditions for it apply (some side effect)

The first part is easy:

DOCTEST_TEST_CASE("MyTest")
{
	auto mock = new MockInterface();
	std::vector<std::unique_ptr<trompeloeil::expectation>> expectations;
	auto init_expectation = NAMED_REQUIRE_CALL(*mock, init());

Now, I want to make the two other tests. Since the above lines are the same for both the tests, I want to reuse code and use DOCTEST_SUBCASE.

DOCTEST_SUBCASE("someMethod") {
	expectations.emplace_back(NAMED_REQUIRE_CALL(*mock, someMethod()));
}

DOCTEST_SUBCASE("otherMethod") {
	expectations.emplace_back(NAMED_REQUIRE_CALL(*mock, otherMethod()));
}

Now I just need a way to invoke the side effects. Unfortunately, there is what I don't know how to do: init_expectation is already defined, and AFAIK, there's no way to attach a side effect after the fact. Could trompeloeil support this? Full example would look like this:

#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN
#include <trompeloeil.hpp>
#include <doctest/doctest.h>

class Interface {
public:
	virtual ~Interface() = default;
	virtual void init() = 0;
	virtual void someMethod() = 0;
	virtual void otherMethod() = 0;
};

class MyApp {
public:
	MyApp(Interface*);
	void exec();
};

class MockInterface : public trompeloeil::mock_interface<Interface> {
	IMPLEMENT_MOCK0(init);
	IMPLEMENT_MOCK0(someMethod);
	IMPLEMENT_MOCK0(otherMethod);
};

DOCTEST_TEST_CASE("MyTest")
{
	auto mock = new MockInterface();
	std::vector<std::unique_ptr<trompeloeil::expectation>> expectations;
	auto init_expectation = NAMED_REQUIRE_CALL(*mock, init());

	DOCTEST_SUBCASE("someMethod") {
		init_expectation.add_side_effect(invoke_side_effect_for_someMethod())
		expectations.emplace_back(NAMED_REQUIRE_CALL(*mock, someMethod()));
	}

	DOCTEST_SUBCASE("otherMethod") {
		init_expectation.add_side_effect(invoke_side_effect_for_someMethod())
		expectations.emplace_back(NAMED_REQUIRE_CALL(*mock, otherMethod()));
	}

	// After setting up all the expectations, actually run the app.
	MyApp app(mock);
	// exec blocks until the app closes.
	app.exec();
}

Some other solutions I've tried:

• put init_expectation inside the SUBCASEs - that would lead would to more code duplication (imagine a lot of nested SUBCASEs where the first line in the same nesting level is copied)
• put the entire SUBCASE tree into .SIDE_EFFECT - haven't tried, but I imagine that the code would be pretty complex and it wouldn't work with the macro
• define an std::function, assign it in the subcases and tell init_expectation to assign it. Like this:

std::function<void()> fn;
auto init_expectation = NAMED_REQUIRE_CALL(*mock, init()).LR_SIDE_EFFECT(fn());

DOCTEST_SUBCASE("someMethod") {
	fn = invoke_side_effect_for_someMethod;
	expectations.emplace_back(NAMED_REQUIRE_CALL(*mock, someMethod()));
}

This would probably do what I would want, but it'd mean, that I would have to define a lot those std::functions in the top scope of the test, and, give them some unique identifier for every SUBCASE that needs the same thing for all of its child SUBCASEs. I could possibly add them to e.g. an std::map, so that I don't have to define them all as separate variables. I haven't tested this, but it seems to me like it would make stuff complicated. However, it seems like the solution, that actually works.

[syyyr]

Also, one more thing: I'm having problems with telling my MyApp to quit. It has an exit() method, so if I could somehow know when all the expectations get fullfilled, I could just call it. Or, if I could attach side effects to existing expectations, I could just do this:

MyApp app(mock);
expectations.back().add_lr_side_effect(app.exit());
app.exec();

[rollbear]

I'm really not very familiar with doctest, and I'm not at all sure how subcases work.

What I think you can do is to create a separate collection of side effects that you want to run, and you set your expectation's .SIDE_EFFECT() to run all of them. Then you can add to, and subtract from, that collection in your subcases.

Something like this:

DOCTEST_TEST_CASE("MyTest")
{
	auto mock = new MockInterface();
	std::vector<std::unique_ptr<trompeloeil::expectation>> expectations;
        std::list<std::function<void()>> side_effects;
	auto init_expectation = NAMED_REQUIRE_CALL(*mock, init())
                                .LR_SIDE_EFFECT(for (auto& f : side_effects) { f(); });

	DOCTEST_SUBCASE("someMethod") {
                auto i = side_effects.insert(side_effects.end(), [&](){ /* blah */ });
                // work
                side_effects.erase(i);
	}

	DOCTEST_SUBCASE("otherMethod") {
                auto i = side_effects.insert(side_effects.end(), [&](){ /* bleh */ });
                // other work
                side_effects.erase(i);
	}

	// After setting up all the expectations, actually run the app.
	MyApp app(mock);
	// exec blocks until the app closes.
	app.exec();
}

For your latter problem, you can query in expectation if it is satisfied or not. See https://github.com/rollbear/trompeloeil/blob/main/docs/reference.md#is_satisfied. Although, I do find it very odd that your test program doesn't know when the unit under test has reached it's conclusion.

[syyyr]

The point of DOCTEST_SUBCASE is that for every leaf subcase, the test is run exactly once, and every iteration it changes the "current" leaf. The advantage is that you can reuse the init/common code outside of the subcases.

What I think you can do is to create a separate collection of side effects that you want to run, and you set your expectation's .SIDE_EFFECT() to run all of them. Then you can add to, and subtract from, that collection in your subcases.

This is more or less what I do in my last code example:

std::function<void()> fn;
auto init_expectation = NAMED_REQUIRE_CALL(*mock, init()).LR_SIDE_EFFECT(fn());

DOCTEST_SUBCASE("someMethod") {
	fn = invoke_side_effect_for_someMethod;
	expectations.emplace_back(NAMED_REQUIRE_CALL(*mock, someMethod()));
}

just with a single function, without a list. It's a fine solution, I don't mind using that, I just thought that maybe there could be a way to somehow set the side effect "in runtime", as in, use .LR_SIDE_EFFECT after the expectation has been created. But I understand, that it might not be possible given how many checks are in the macros (and I guess they are compile time checks).

For your latter problem, you can query in expectation if it is satisfied or not. See https://github.com/rollbear/trompeloeil/blob/main/docs/reference.md#is_satisfied.

I can't do this because after I run .exec() I can only run code inside side effects.

Although, I do find it very odd that your test program doesn't know when the unit under test has reached it's conclusion.

The way my program works is that I run it via .exec(), then it starts listening for events, and responds to them. The app itself doesn't know that it's being tested. The tests do not do anything until after I run .exec(), because the test code is in the side effects, and expectations. After all the expectations are satisfied, the app continues to run, because no one told it, it should stop. Another idea: maybe trompeloeil coud expose some sort of a callback mechanism, that would notify me, whenever an expectation is completed?

This is maybe bending trompeloeil (and doctest) too much, and my method of testing might be a little weird. Unfortunately, I haven't found of a better way to test apps that have an event loop, than define a bunch of expectations and attach side effects beforehand, and then run the app and let it do its thing. But of course I understand that figuring out how to test my app isn't in the scope of this project.

[rollbear]

It's doable, I think, but it seems like a lot of work for a very special case that is quite easy to achieve anyway.

[syyyr]

Okay, no worries. I thought it wouldn't be so easy. I'm closing this, but feel free to reopen, if you want to track this somehow.