Decision and Condition coverage

[xd009642]

Problem statement

As a fairly big feature to introduce into tarpaulin I'm creating an issue to track things that need to be investigated/implemented/solved to achieve decision (a.k.a branch) coverage and condition coverage.

Decision coverage can be achieved purely through source analysis and tagging lines as the first instruction in a branch, however condition coverage requires the evaluation of boolean subconditions. Given the boolean expression A || (B && C), decision coverage checks that the whole thing is evaluated true and false whereas condition coverage determines that A, B and C have all evaluated as true and false.

Because of this pure source analysis won't completely solve the issue. Instead the solution will be a mixture of source analysis and peeking into program registers/instructions.

Tasks

• [x] Identify branches and lines containing conditions via source analysis or DWARF
• [ ] Identify number of boolean subconditions
• [ ] Figure out how to evaluating ASM for the subconditions
  • [ ] Short-circuiting conditions need to be fully stressed but as an initial false future subconditions might cause run-time errors
  • [ ] Optimisations could remove subconditions - should these be tested?
  • [ ] How to evaluate ASM expressions without affecting test execution
    • [ ] Can ASM be used to generate a dynamic lib which is dynamically loaded and used to get condition state?
• [ ] Identify how to handle different rust constructs
  • [ ] generics (they're really just a branch with the number of branches determined at compile-time)
  • [ ] iterator functions (filter, fold, map etc.)
  • [ ] pattern matching
  • [ ] ? operator
  • [ ] while and if let
  • [ ] exceptions
  • [ ] macros

New dependencies

This issue might need x86_64 disassembly. With this in mind https://github.com/gz/rust-x86 looks like a good potential crate.

Resources

• Nasa guide on MC/DC

[marco-c]

Have you investigated using the experimental gcov profiling feature? It would give you decision coverage for free, and it supports any target.

[xd009642]

I'm not sure how this would work in practice given that gcov instruments and traces the code itself. It's always available as an alternative coverage tool as there's nothing stopping people using gcov on rust code but that fact it's got no understanding of rust will probably result in mistakes when covering things like match arms or dealing with generic code.