cpputest
Parameterized tests
I'd be happy with an example of how to do it. Google can't seem to find one for me. :)
@jbrains Uhm, well it isn't supported yet, so I'm a bit unclear what you mean with an example :)
@basvodde Then I suppose that gives me my answer. I'm glad I didn't spend too long looking. :)
@jbrains Eheh, yeah... what were you thinking. Asking for examples of features that don't exist :P It almost sounds like...
Specification by Example :D
I have missed in actual work the ability to loop tests, e.g. run a test case with a loop counter.
@basvodde Is that by what you meant, by any chance?
e.g.
LOOP_TEST(GroupName, TestName, n, 0, 1, 100)
{
dosomething(n);
CHECK(some_condition);
}
A prameterized test would be something like this:
TEST(x, y, z) { CHECK(do_stuff(y)) }
TEST_PARAMETERIZED(x, y, 1, 2, 3, 4, 5, 6)
Now it would create tests for all these values...
The above won't ever work, but it is something to think about :)
There is a TestTestingFixture class implemented, but ... how shall I use it? So... there is an example what I like to do: I'd like to test a function what implements a simple state machine:
void periodicallyCalled_cookingStateMachine()
{
static SimpleString currentMeal = "";
static State state = Pending;
static unsigned remainingTime = 0;
switch (state) {
case Pending:
currentMeal = getNextOrder();
if (!currentMeal.IsEmpty()) {
remainingTime = getCookTime(currentMeal);
if (remainingTime > 0) {
startCooking();
state = Cooking;
} else {
serve();
state = Pending;
}
}
break;
case Cooking:
if (remainingTime > PERIOD_TIME) {
remainintTime -= PERIOD_TIME;
} else {
serve();
state = Pending;
}
break;
}
reportCurrentState(state);
}
In the example above the required mocks are: getNextOrder, getCookTime, serve, reportCurrentState (startCooking is a local static function)
A well designed test of this state machine contains TestGroups for transitions grouped by start state. This is a good idea, because in this case the setup function can hold the required steps to reach the start state (it makes a possibility to differentiate failures by it comes from a prepare step or from the main transition). A TestCase (what tests a transition) contains TestFixtures for making boundary test:
/* Input and expected values */
static struct {
bool expect_getNextOrder;
SimpleString mealName;
bool expect_getCookTime;
unsigned cookTime;
bool expect_serve;
State state;
} parameters;
static void resetParameters()
{
parameters.expect_getNextOrder = false;
/* TODO: set mailName to a random one */
parameters.expect_getCookTime = false;
/* TODO: set cookTime to a random one */
parameters.expect_serve = false;
parameters.state = Pending;
}
static void setExpectations()
{
if (parameters.expect_getNextOrder)
mock().expectOneCall("getNextOrder").andReturnValue(parameters.mealName);
if (parameters.expect_getCookTime)
mock().expectOneCall("getCookTime").withStringParameter("mealName", parameters.mealName).andReturnValue(parameters.cookTime);
if (parameters.expect_serve)
mock().expectOneCall("serve").withStringParameter("mealName", parameters.mealName);
mock().expectOneCall("reportCurrentState").withIntParameter("state", parameters.state); /* TODO: its an enum and not an int */
}
/* State transitions */
static void Step_PendingToCooking(const std::string &mealName, const unsigned &cookTime) {
resetParameters();
parameters.expect_getNextOrder = true;
parameters.mealName = mealName;
parameters.expect_getCookTime = true;
parameters.cookTime = cookTime;
parameters.state = Cooking;
setExpectations();
periodicallyCalled_cookingStateMachine();
mock().checkExpectations();
}
static void Step_PendingToPending(const std::string &mealName) {
resetParameters();
parameters.expect_getNextOrder = true;
parameters.mealName = mealName;
parameters.expect_getCookTime = true;
parameters.cookTime = 0;
parameters.state = Pending;
setExpectations();
periodicallyCalled_cookingStateMachine();
mock().checkExpectations();
}
static void Step_CookingToPending() {
// /* Wait cook time */
// unsigned waitCycles = parameters.cookTime / PERIOD_TIME;
// for(unsigned i = 0; i < waitCycles; ++i) {
// resetParameters();
// parameters.state=Cooking;
// setExpectations();
//
// periodicallyCalled_cookingStateMachine();
//
// mock().checkExpectations();
// }
/* The real transition */
resetParameters();
parameters.expect_serve=true;
parameters.state=Pending;
setExpectations();
periodicallyCalled_cookingStateMachine();
mock().checkExpectations();
}
TEST_GROUP(TransitionsFrom_Pending) {
void setup() {}
void teardown() { mock().clear(); }
};
TEST_GROUP(TransitionsFrom_Cooking) {
void setup() {
Step_PendingToCooking("myMeal", 1);
}
void teardown() { mock().clear(); }
};
TEST(TransitionsFrom_Pending, ToCooking) {
Step_PendingToCooking("Boiled Egg", 1);
};
TEST(TransitionsFrom_Pending, ToPending) {
Step_PendingToPending("Fresh Salad");
};
TEST(TransitionsFrom_Cooking, ToPending) {
Step_CookingToPending();
}
Fixtures shall be used in test TransitionsFrom_Pending/ToCooking. I'd like to call Step_PreparedToCooking with different cook time (like 1, 0, UINT_MAX, PERIOD_TIME-1, PERIOD_TIME, PERIOD_TIME+1, ...). All of my test case use "parameters" as TestFixture, but with current framework there is no way to add multiple fixtures to a TEST. My proposal what looks similar than the current style of the framework:
TEST_WITH_FIXTURES(TransitionsFrom_Prepared, ToCooking)
{
TEST_BODY() {
unsigned cookTime = mock().getCurrentFixture().getUnsignedParam("cookTime");
Step_PreparedToCooking(cookTime);
}
TEST_GENERATE_FIXTURES() {
mock().getCurrentTest().addFixture("Normal").withUnsignedParam("cookTime", 1);
mock().getCurrentTest().addFixture("Zero").withUnsignedParam("cookTime", 0);
mock().getCurrentTest().addFixture("Max").withUnsignedParam("cookTime", UINT_MAX);
}
};
And when a UTest(Shell) gets running, it shall iterate through internal fixtures (by TestFixture::_next member). (TEST_FIXTURES can be identified by a name, can be extended with "result shall be passed or failed"...)
There shall be a possibility to "reuse" a fixture. I mean when cooking time equals with PERIDO_TIME multiplied by 10, (I'd like to call "Stay_Cooking" 9th times with same fixture (parameters.state=Cooking) and after that once with a different one. To solve this issue, I think a wait loop shall be added to the beginning of Step_CookingToPending implementation - check commented lines. Do you have a better idea?)
Note: The code above contains a lot of compile errors/warnings I think, because I did not try to build it. Sorry.
Should CppUTest support Parameterized tests better?
Yes please!
I'm used to have parametric tests in other frameworks/languages, for example:
