[infra] Indicate coverage run failure when libFuzzer reports an error

[phi-go]

Currently OOM's and timeouts for individual corpus entries during coverage measurement are silently ignored, as libFuzzer still exits with a zero returncode. This PR adds a grep for error messages during the coverage run. Additionally, add a small log file is created to indicate the fuzzer target for which the coverage measurement did not complete succesfully.

[DavidKorczynski]

/gcbrun trial_build.py --sanitizer coverage --fuzzing-engine libfuzzer

[phi-go]

@DavidKorczynski I think something went wrong with the command?

Starting Step #1
Step #1: Already have image: gcr.io/oss-fuzz-base/trial-build
Step #1: INFO:root:Command: ['--sanitizer', 'coverage', '--fuzzing-engine', 'libfuzzer', '--repo', 'https://github.com/phi-go/oss-fuzz', '--branch', 'push-qnznzrmxrksp'].
Step #1: usage: /opt/oss-fuzz/infra/build/functions/gcb.py [-h]
Step #1:                                                   [--sanitizers SANITIZERS [SANITIZERS ...]]
Step #1:                                                   [--fuzzing-engines FUZZING_ENGINES [FUZZING_ENGINES ...]]
Step #1:                                                   [--repo REPO]
Step #1:                                                   [--branch BRANCH]
Step #1:                                                   [--force-build]
Step #1:                                                   projects [projects ...]
Step #1: /opt/oss-fuzz/infra/build/functions/gcb.py: error: the following arguments are required: projects
Finished Step #1
ERROR
ERROR: build step 1 "gcr.io/oss-fuzz-base/trial-build" failed: step exited with non-zero status: 2

[DavidKorczynski]

/gcbrun trial_build.py all --sanitizer coverage --fuzzing-engine libfuzzer

[phi-go]

@DavidKorczynski it seems the branch was not up to date

Needs /gcbrun(https://github.com/google/oss-fuzz/commit/b094c0b06c060459bbcd89b00d662c8bc7f54f6c) to run latest commit

[DavidKorczynski]

/gcbrun trial_build.py all --sanitizer coverage --fuzzing-engine libfuzzer

[phi-go]

Seems this one timed out before doing any fuzzing or coverage measurements?

[DavidKorczynski]

Will run one more trial build -- please don't submit new patches in the meantime, if the trial build succeeds then this should be good to go

[DavidKorczynski]

/gcbrun trial_build.py all --sanitizer coverage --fuzzing-engine libfuzzer

[phi-go]

All errors seem to be a missing corpus. Unfortunately these errors seem to be unrelated to this change, I don't think I can really do anything here.

Starting Step #25 Step #25: Already have image (with digest): gcr.io/oss-fuzz-base/base-runner-testing-push-qnznzrmxrksp Step #25: warning [/corpus/fuzzer.zip]: zipfile is empty Step #25: Failed to unpack the corpus for fuzzer. This usually means that corpus backup for a particular fuzz target does not exist. If a fuzz target was added in the last 24 hours, please wait one more day. Otherwise, something is wrong with the fuzz target or the infrastructure, and corpus pruning task does not finish successfully. Step #25: ******************************************************************************** Step #25: Code coverage report generation failed. Step #25: To reproduce, run: Step #25: python infra/helper.py build_image wavpack Step #25: python infra/helper.py build_fuzzers --sanitizer coverage wavpack Step #25: python infra/helper.py coverage wavpack Step #25: ******************************************************************************** Finished Step #25 ERROR ERROR: build step 25 "gcr.io/oss-fuzz-base/base-runner-testing-push-qnznzrmxrksp" failed: step exited with non-zero status: 1

or

Starting Step #25 Step #25: Already have image (with digest): gcr.io/oss-fuzz-base/base-runner-testing-push-qnznzrmxrksp Step #25: warning [/corpus/fuzz_demangle.zip]: zipfile is empty Step #25: Failed to unpack the corpus for fuzz_demangle. This usually means that corpus backup for a particular fuzz target does not exist. If a fuzz target was added in the last 24 hours, please wait one more day. Otherwise, something is wrong with the fuzz target or the infrastructure, and corpus pruning task does not finish successfully. Step #25: ******************************************************************************** Step #25: Code coverage report generation failed. Step #25: To reproduce, run: Step #25: python infra/helper.py build_image glog Step #25: python infra/helper.py build_fuzzers --sanitizer coverage glog Step #25: python infra/helper.py coverage glog Step #25: ******************************************************************************** Finished Step #25

[phi-go]

Actually, it seems these missing corpus are a larger issues. Spot checking projects on the fuzz-introspector overview they all seem to fail building coverage.

[DavidKorczynski]

Actually, it seems these missing corpus are a larger issues. Spot checking projects on the fuzz-introspector overview they all seem to fail building coverage.

yeah, there has been some problems with this as of late.

Thanks again for this!