google
bad_build_check didn't catch latest Jazzer breakage
Context: https://github.com/google/oss-fuzz/pull/8275
bad_build_check should've caught this and prevented builds from getting uploaded.
@jonathanmetzman
The Jazzer bug caused a crash if the fuzz target wasn't executed with a -seed argument. Both the bad build check and all our internal tests did that for the sake of reproducibility.
Not sure what to do about this particular bug type - we now have an internal one covering it.
Could we perhaps add one extra check (for libFuzzer engines) where we:
- Try running with -runs=1
- Try reproducing with an empty input
None of these should result in a crash if it's a good build.
Could we perhaps add one extra check (for libFuzzer engines) where we:
- Try running with -runs=1
- Try reproducing with an empty input
None of these should result in a crash if it's a good build.
IMO the check worked as it should. There was nothing wrong with Jazzer projects and reporting the bugs there instead of when we update Jazzer is not great UX. Ideally we wcould make seperate tests that run during updating
Could we perhaps add one extra check (for libFuzzer engines) where we:
- Try running with -runs=1
- Try reproducing with an empty input
None of these should result in a crash if it's a good build.
IMO the check worked as it should. There was nothing wrong with Jazzer projects and reporting the bugs there instead of when we update Jazzer is not great UX. Ideally we wcould make seperate tests that run during updating
If we don't pin, yes.
If we had these bad build checks and we pin, then we can catch them with our trial_build checks right?
Even ignoring Jazzer, making sure a target doesn't crash with an empty input is pretty valid right?
Even ignoring Jazzer, making sure a target doesn't crash with an empty input is pretty valid right?
I guess. I think our existing bad build check probably cover this 99% of the time and just happened to miss it here.
Indeed. Let's:
- Keep pinning Jazzer
- Add a test to make sure that running against with an empty input does not crash in some way to
bad_build_check.
@fmeum you mentioned being able to help with this. Would you be able to help us with a PR here? Thanks!
@oliverchang With empty input, do you mean an invocation of the form fuzz_target empty_file? If so, that would have caught the issue and I can add it in a PR.
@oliverchang With empty input, do you mean an invocation of the form
fuzz_target empty_file? If so, that would have caught the issue and I can add it in a PR.
Yep!