write better documentation

[dvyukov]

Topics that need to be covered:

• how to write good Fuzz functions (checking for logical bugs, cross checking different implementations, examining as much code as possible, testing functions that require more complex inputs and not just []byte)
• how to use several Fuzz functions in a single package
• corpus collection

[dvyukov]

Also:

• what to do with known non-fixed bugs
• how to deal with inputs that are not "suitable" for program (e.g. input must be longer than N, or doesn't contain particular bytes)

[mimoo]

hey, I'd be interested about more documentation on how to use -procs and -worker

What's a good value for -procs? How to figure that out?

EDIT: looks like it takes the min between the config and the actual number of processors you have

runtime.GOMAXPROCS(min(*flagProcs, runtime.NumCPU()))

[dvyukov]

@mimoo both of the flags have default value. Default value is the good value (what would it be otherwise? :))

[mimoo]

Really? I went from <100 exec/s to >3000 exec/s by trying -procs 10 instead of the default.

[dvyukov]

So how did you figure value 10 out? What's the default value for -procs on your machine? Do you have any insight as to why 10 is significantly faster?

[mimoo]

I just started looking into it, I randomly put the number 10 and got huge numbers. I just checked and numProcs() returns 2 on my linux machine. My guess is the program I'm fuzzing is slow anyway, so having multiple instances is increasing the speed, but I'm not really sure. Any hints on how I can investigate this? (getting similar improvements on a macbook with 4 procs.)

[dvyukov]

The only thing I can think of is if the program is doing some IO and is not CPU bound, then increasing -procs should help. Is your test doing any IO? Or merely computations?

[mimoo]

It should not do any IO, but there are a good amount of goroutines (you should print out the runtime.numGoRoutines() in that output btw :D)

[dvyukov]

Then I am out of ideas. If you profile it, maybe we can get some hints. Otherwise we could say "you can try to shuffle the value randomly until you get better results", but this does not look much better than saying nothing.

[mimoo]

Profiling doesn't really yield much, but I managed to optimize it via this investigation so thanks.

Another question, my fuzzing is becoming gradually slow until it reaches the double digits after hours. Any idea what might cause that on go-fuzz side? (I lose around 20 exec/s every 10 minutes)

Or any idea on how I could investigate that? I'm thinking maybe returning a memprofile of the fuzz every hour would help :/ ?

[dvyukov]

Please post excerpts from go-fuzz output with -v=1 flag (in the beginning when it's fast, in the middle and when it's slow). Also a cpu profile when it's slow can be helpful.

[prologic]

Another point that's come to mind in my playing around with go-fuzz

When do you consider fuzzing complete? Does go-fuzz eventually give up?

[dvyukov]

@prologic Formally, it's never complete because there is infinite amount of input variations. If you are doing a one-off run, then I would say a weekend without crashes it good enough. However, I've seen a bug in OpenSSL IIRC which took years of fuzzing time to discover. But with a continuous fuzzing deployment (ideal) one does not need to answer this question because new changes come in faster anyway.