jonathanmetzman issues

Results 243 issues of


                                            jonathanmetzman

Add method for verifying bugs in benchmarks

We are going to add benchmarks in which OSS-Fuzz found many bugs as part of bug-based benchmarking. We need a method of verifying these crashes reproduce in the fuzzbench builds...

Add progress bars to reports.

A nice to have feature would be progress bars for measurement and trials on reports. This could give an idea of when an experiment is done. I think it should...

enhancement

Support running FuzzBench without Google Cloud

Local experiments are currently not truly local because they depend on Google Cloud Storage (GCS). We should break this dependency. We use gsutil which has bash-like commands for managing files...

priority

Verify that clang cov and sancov produce similar results on the same experiment

You might need to do some hacking to make sure it downloads the right coverage binaries, but you can probably do this by running measurer.py standalone and deleting (save them...

priority

needs analysis

Variance in FuzzBench trials with same random seed

@vanhauser-thc wrote [here](https://github.com/google/fuzzbench/issues/654#issuecomment-672845457): >the experiment has started => https://www.fuzzbench.com/reports/2020-08-12/index.html and we can see the aflplusplus_same1 .. aflplusplus_same3 instances to have quite a different coverage. >for several benchmarks this will even...

needs analysis

AssertionError in makeUniformBindings on master branch

I get this crash pretty consistently: To repro: ``` $ java -ea -cp ~/graphicsfuzz-build/jar/tool-1.0.jar com.graphicsfuzz.generator.tool.GenerateShaderFamily ~/shaders-build/references/GLSL_300_es/nested_loops.json ~/shaders-build/donors /tmp/k --generate-uniform-bindings 11:42:10.430 INFO ExecHelper - Result: 0 java.lang.AssertionError at com.graphicsfuzz.common.transformreduce.GlslShaderJob.makeUniformBindings(GlslShaderJob.java:119) at com.graphicsfuzz.generator.tool.PrepareReference.prepareReference(PrepareReference.java:142)...

Gitlab instructions do not just work.

I had to make [this config file](https://gitlab.com/metzman/cfl-gitlab/-/blob/ea4d5f97e13196b29b67ddcba88784fa9981a26b/.gitlab-ci.yml) which borrowed from https://gitlab.com/securitykernel/cflite-example/-/blob/main/.gitlab-ci.yml#L7 because the docs were incomplete. CC @catenacyber

jonathanmetzman