iAI icon indicating copy to clipboard operation
iAI copied to clipboard

Unit Tests compareResults numeric ID based ground truth values are fragile

Open blipper opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe.

A typical test pattern in unittets is TEST_F(IFDSTaintAnalysisTest, TaintTest_04) { initialize({PathToLlFiles + "dummy_source_sink/taint_04_cpp_dbg.ll"}); IFDSSolver_P<IFDSTaintAnalysis> TaintSolver(*TaintProblem); TaintSolver.solve(); map<int, set> GroundTruth; GroundTruth[19] = set{"18"}; GroundTruth[24] = set{"23"}; compareResults(GroundTruth); }

Specifying individual IDs is very fragile since it depends on how clang compiles and a single change can break everything.

Describe the solution you'd like A more flexible/less fragile pattern

e.g. specify the instruction pattern (with filters/qulifiers) %kIVSize = alloca i32, align 4, !psr.id !35 | ID: 7

Something like IDLookup.Builder.IRpattern("%kIVSize = alloca i32, align 4").inFunction("main").afterIRpattern("%p = alloca %struct.Params").build().getID();

blipper avatar Apr 18 '21 21:04 blipper

I agree that this is fragile. So far we have not had problems with that, which I don't mean as an excuse not to do it right.

MMory avatar Apr 26 '21 11:04 MMory

One example is the differences than result between stdlibc++ and libcxx

blipper avatar Jul 26 '21 15:07 blipper