Generate runnable example for crashers?

[tv42]

This would make reproducing the bug for debugging, and submitting upstream, pretty easy. A few alternatives come to mind:

A. Create a new fuzz_<hash>_test.go, embedding contents of fuzz.go except func Fuzz renamed for uniqueness:

package pkgname

func Fuzz<hash>(data []byte) int {
...
}

func TestFuzz<hash>(t *testing) {
    Fuzz<hash>([]byte("contents of crashers/x.quote here"))
}

(if user defined helper functions in fuzz.go, this can still result in name conflicts)

B. Assuming upstream has the fuzz framework in place, just write a new command. Running it will require e.g. go run -tags gofuzz workdir/crashers/<hash>.main.go.

package main

import "github.com/jdoe/system-under-test/pkgname"

func main() {
    pkgname.Fuzz([]byte("contents of crashers/x.quote here"))
}

C. If upstream doesn't have func Fuzz in place, it might make more sense to generate a whole standalone copy of fuzz.go with a func main() embedded in it. Adjusting imports makes this more difficult, as the typical idiom seems to be to put func Fuzz in the same package as the system under test (and that's needed for fuzzing internal functions).

[dvyukov]

I agree that would be useful. But I am not sure what is the right solution. As you noted several solutions possible, all have some downsides. If you can work out a solution that works in all contexts, then I am happy to accept a pull request.

[tv42]

I just stumbled back to this (sorry for not doing much more than a drive-by omgponies request), and gave it some more thought.

C. seems impractical and/or impossible.

B. is brittle, as the Fuzz function might change

A. seems like the best idea.

Let's consider this ticket to be about alternative A only.

[landaire]

I just stumbled upon this as well and I think I kind of created what you're looking for today: https://github.com/landaire/go-fuzz-issue-generator

I wrote it in about an hour just to save myself some time from formatting ~100 reports, but the gist is you give it a template to use and the crash file and it spits out a runnable program with its output. It is not pretty by any means and still requires some manual effort... and the readme is somewhat confusing.

I used it today like so (that glob is a zsh glob to get files without suffixes):

❯ for f in ~/go/src/fuzzing/gotoml/crashers/*[::alnum::]; do ./gfig ~/go/src/fuzzing/gotoml/template.go $f; done

Not an ideal solution but you get the idea.

[dgryski]

Maybe having an external tool similar to "go tool bug" is the right approach here.

[dvyukov]

The proposal at https://github.com/golang/go/issues/19109#issuecomment-285456008 includes -fuzzinput input to run a single input (will be needed for continuous fuzzing integration (e.g. OSS-Fuzz) to minimize, bisect, re-test, etc). Assuming that build will be fast enough, I think -fuzzinput input can be used manual debugging/testing as well. What do you think?

[dgryski]

Ah yes, missed the reduction in scope to simply adding/running a single test case for a package that already has a full fuzzing infrastructure. I think it will be addressed sufficiently by -fuzzinput.