oss-fuzz icon indicating copy to clipboard operation
oss-fuzz copied to clipboard

Better pre-commit monitoring for MSan false positives

Open davidben opened this issue 1 year ago • 0 comments

After the most recent round of MSan false positives (https://github.com/google/oss-fuzz/issues/11886), perhaps OSS-Fuzz could consider better pre-commit testing? For example:

Before every tooling update, run the fuzzers over the existing corpus, or at least a random sample of them. Collect the new bugs that would be filed and the bugs that would be closed. If there are a lot, someone from OSS-Fuzz should triage them and confirm that this is working as intended and there isn't something gone horribly wrong.

If triaging other projects things is tricky (pretty understandable), OSS-Fuzz could maintain a benchmark project containing fuzzers that are known to surface problems and not surface problems. Those could be used for pre-commit testing. In particular, there should be benchmark fuzzers that break if libc and libc++ are not correctly instrumented in MSan.

Then, just as well-run projects are expected to write regression tests when things break, OSS-Fuzz should add to this benchmark project whenever a false positive slips through.

davidben avatar May 10 '24 19:05 davidben