stc icon indicating copy to clipboard operation
stc copied to clipboard

Ease of understanding metrics

Open robpalme opened this issue 2 years ago • 1 comments

The four classes of metric for assessing completion are not easy to understand unless you go to the website and read the definition. Especially in tweet form for newcomers.

  • Required means a true negative, which is a type error that is not reported by stc.
  • Matched means a true positive, which is a type error that is reported by stc and correct.
  • Extra means a false positive. In other words, it's the number of incorrect errors stc emits while it should not.
  • Panic means the analyzer panicked while validating input files, due to a logic bug.

It's hard to know which of those numbers you want to go up or down. So in string form I suggest ordering and calling them:

  1. matched (was matched)
  2. bad_missed (was required)
  3. bad_excess (was extra)
  4. bad_panic (was panic)

robpalme avatar Jan 30 '23 11:01 robpalme

I'll use those words starting from the next update. Thank you!

kdy1 avatar Feb 03 '23 13:02 kdy1