Laszlo Szekeres

Good point! Another question is how to indicate this on the coverage growth plot. One option would be to start the Y axis at seed coverage instead of 0, the...

> What are path or state transitions? I think that refers to a "coverage point" (eg. edge / block / [region](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html) / or what have you).

> Great idea! I suppose this would show the coverage overlap (difference) between fuzzers? Might help with debugging a fuzzer. Note that the coverage overlap/difference between fuzzers is already shown...

Thanks a lot! It looks like this is two relatively independent changes: ie. (1) the new experiment summary table, and (2) the new fuzzer oriented template. Would it make sense...

Thanks for the detailed explanation, and sorry for the slow response! > Of the new charts, I find the "Trial Ranks" chart to be use most useful, but maybe not...

> Does this matter for Fuzzbench data? I did some quick analysis on 2020-11-01 experiment just to check Scipy versus R. I found a couple changes in the significance level...

> > combine all clones in RUN and prune to save space > > Is this really a big concern? We don't force this in OSS-Fuzz. i don't want to...

Thanks for looking into this! Would this change be consistent with the definition of "critical difference" in https://www.jmlr.org/papers/volume7/demsar06a/demsar06a.pdf ?

Thanks! My translation from the paper's terminology to fuzzbench's terminology is the following: - algorithm, classifier => fuzzer (e.g., afl) - data set => benchmark (e.g., libxml) - performance score...

> the proposed alternative would change the "translation" like this: > > data set => fuzzing campaign on benchmark (e.g., libxml) Right. I don't think we can do that. Doing...