FLiT
FLiT copied to clipboard
Allow multiple test points in a single test case
Currently, you can return a single string or floating-point result at the end of each test case. This is analogous to a unit testing framework that only allows a single assertion at the end in the form of a single Boolean return value.
This enhancement opportunity is analogous to having multiple assertions in a test case. The idea is to allow to have multiple "comparison points" similar to assertions. Something like the following:
virtual void
run_impl(const flit::TestInput<T>& ti, const flit::GroundTruth<T>& gt) override {
T a = ti.vals[0], b = ti.vals[1], c = ti.vals[2];
flit::COMPARE("first", a, gt, [](T x, T y) { return x - y; });
flit::COMPARE("second", b, gt, [](T x, T y) { return std::abs(x - y); });
flit::COMPARE("third", c, gt, [](T x, T y) { FLIT_UNUSED(y); return x; });
flit::COMPARE("fourth-string", "some-string", gt, [](std::string x, std::string y) {
return x.size() - y.size();
});
}
Then this would result in four entries into the database for each time this test is executed. Each of these entries is a "test point" (similar to an assertion) that is named by the first argument to flit::COMPARE
. The calls to flit::COMPARE
would replace the return value. Also the compare()
method in the class would be removed and instead function pointers (or lambda expressions) would be used in each flit::COMPARE
call. Supported types would be the same: float
, double
, long double
, and std::string
.
This is at least the idea. What do others think?