oss-fuzz icon indicating copy to clipboard operation
oss-fuzz copied to clipboard

Regression range should include OSS-Fuzz infrastructure changes

Open davidben opened this issue 1 year ago • 4 comments

It's happened a few times now that OSS-Fuzz infrastructure changes have caused MSan false positives in the fuzzers. Most recently, https://github.com/google/oss-fuzz/pull/11880, but other times we've been hit by regressions in Clang, etc.

When this happens, it is difficult to diagnose on the projects' side, and we often end up spending a lot of time trying to reproduce the issue. (It takes me less time now, but only because I know to be suspicious of OSS-Fuzz MSan reports. That's not exactly a good outcome either! :grin: )

Could the regression range in OSS-Fuzz reports include changes to the compiler and other infrastructure? That way we could look at recent changes there if the regression range for the project itself doesn't check out.

davidben avatar May 01 '24 18:05 davidben

Sorry for the false positives. This seems like a fair request. @oliverchang I think this is a good first issue for some of our new teammates. WDYT?

jonathanmetzman avatar May 01 '24 18:05 jonathanmetzman

Oh, don't worry about it! Stuff breaks and MSan is really, really finicky. Meh. :smile: This was just a thought to help pinpoint things, since I otherwise have no idea how to tell what infra stuff changed recently.

davidben avatar May 01 '24 18:05 davidben

This was just a thought to help pinpoint things

Another idea I had was to reproduce msan crashes in valgrind. If valgrind can not reproduce the uninitialized read, it is more likely to be correct than not. See also #5836

maflcko avatar May 02 '24 14:05 maflcko

That will end up suppressing true positives. valgrind works by instrumenting the compiled code and can't see uninitialized memory that happened to not survive through the compiler's optimization passes. It also needs to conservatively assume many things are fine, because it needs to accept code that compilers output.

For example, valgrind allows you to compute over uninitialized memory, as long as you don't branch on it, but C/C++'s rules are actually much stricter.

davidben avatar May 02 '24 15:05 davidben