klee icon indicating copy to clipboard operation
klee copied to clipboard

Add more statistics (incl. branches and termination classes)

Open 251 opened this issue 3 years ago • 1 comments

This PR adds the following statistics:

  • InhibitedForks
  • QCacheHits/Misses
  • ExternalCalls
  • Allocations (this one exposes the existing one, in other projects I just count calls to executeAlloc)
  • renames States to ActiveStates (active), adds States (total)
  • renames numQueries/Queries to SolverQueries (into constraint solver), adds Queries (into solver chain)
  • branch types
  • termination classes

Notes:

  • The termination class commit adds three new termination functions to cover all classes (A "class" categorises termination types. I don't want to expose all termination types as we have quite a lot.) and adds two classes (EarlyUser and EarlyAlgorithm).
  • I've replaced usages of terminateOnError with terminateOnProgramError to reflect the termination class and be more consistent
  • there are some minor reformattings in the StatsTracker
  • I added some gaps in the enumeration of the termination types to make it easier to add new types

Update: I had to remove the 4GB allocation for the model error. MSAN couldn't handle it in the CI.

Summary:

Checklist:

  • [x] The PR addresses a single issue. If it can be divided into multiple independent PRs, please do so.
  • [x] The PR is divided into a logical sequence of commits OR a single commit is sufficient.
  • [x] There are no unnecessary commits (e.g. commits fixing issues in a previous commit in the same PR).
  • [x] Each commit has a meaningful message documenting what it does.
  • [x] All messages added to the codebase, all comments, as well as commit messages are spellchecked.
  • [x] The code is commented OR not applicable/necessary.
  • [x] The patch is formatted via clang-format OR not applicable (if explicitly overridden leave unchecked and explain).
  • [x] There are test cases for the code you added or modified OR no such test cases are required.

251 avatar Jan 11 '22 19:01 251

Codecov Report

Merging #1465 (8ad8edf) into master (91468cf) will increase coverage by 0.12%. The diff coverage is 84.61%.

:exclamation: Current head 8ad8edf differs from pull request most recent head a1eca15. Consider uploading reports for the commit a1eca15 to get more accurate results

@@            Coverage Diff             @@
##           master    #1465      +/-   ##
==========================================
+ Coverage   70.27%   70.40%   +0.12%     
==========================================
  Files         155      156       +1     
  Lines       18899    18946      +47     
  Branches     4442     4447       +5     
==========================================
+ Hits        13282    13338      +56     
+ Misses       3660     3658       -2     
+ Partials     1957     1950       -7     
Impacted Files Coverage Δ
lib/Core/Executor.h 77.27% <ø> (ø)
lib/Solver/DummySolver.cpp 33.33% <25.00%> (ø)
lib/Core/CoreStats.cpp 40.00% <40.00%> (ø)
lib/Core/Executor.cpp 76.69% <80.35%> (+0.42%) :arrow_up:
lib/Core/SpecialFunctionHandler.cpp 71.96% <88.88%> (ø)
lib/Core/ExecutionState.h 84.00% <100.00%> (+0.66%) :arrow_up:
lib/Core/ExternalDispatcher.cpp 87.31% <100.00%> (+0.09%) :arrow_up:
lib/Core/MergeHandler.cpp 91.66% <100.00%> (ø)
lib/Core/StatsTracker.cpp 82.02% <100.00%> (+0.38%) :arrow_up:
lib/Core/TimingSolver.cpp 83.33% <100.00%> (+1.51%) :arrow_up:
... and 4 more

... and 2 files with indirect coverage changes

codecov[bot] avatar Jan 11 '22 19:01 codecov[bot]

Sure but please do not merge before fixing the CI.

251 avatar Mar 21 '23 18:03 251

@251 Can you rebase this one? It's time to merge this.

MartinNowack avatar Mar 23 '23 14:03 MartinNowack

@ccadar Looks like a great in-depth documentation update in: https://github.com/klee/klee.github.io/pull/348. I guess there is no reason to block this anymore.

MartinNowack avatar Mar 23 '23 14:03 MartinNowack

@251 thanks for the updating the documentation!

ccadar avatar Mar 23 '23 17:03 ccadar