goconvey icon indicating copy to clipboard operation
goconvey copied to clipboard

Support Go 1.18 fuzz feature

Open srabraham opened this issue 3 years ago • 3 comments

There's now a fuzzing feature to Go testing: https://go.dev/doc/fuzz/ It's a bit different than benchmarking, since these are real tests that have assertions in them. GoConvey seems like a good framework to consider.

It's not currently pretty using this feature with GoConvey though. Beyond any other UX questions, the most obvious thing is that you don't start with a *testing.T, but rather a *testing.F. That means that the top-level Convey call needs to be done inside the f.Fuzz, which is kind of weird. Anyway, properly supporting fuzzing everywhere in GoConvey would probably take some effort, but it's worth considering.

func FuzzSomethingSilly(f *testing.F) {
	f.Add(0)
	f.Add(1)
	f.Add(2)
	f.Fuzz(func(t *testing.T, n uint64) {
		Convey("For fuzz input", t, func() {
			So(n, ShouldEqual, n)
		})
	})
}

srabraham avatar Apr 20 '22 15:04 srabraham

@riannucci are you pretty much the owner of this project now? It's good to see you again! 😄

srabraham avatar Apr 20 '22 15:04 srabraham

Lol yep, I'm kind of the de-facto maintainer at this point... I'm unfortunately not able to give the project the attention it deserves :( (if you wanna pitch in, please do so!)

Yeah I wonder if we can use testing.TB instead of *testing.T in the convey interfaces? That might be sufficient?

riannucci avatar Apr 20 '22 18:04 riannucci

(and yeah there's a whole bunch of UX questions... I feel like Convey is due for a rework on how it internally tracks test state and how it does stdio, too, in order to take advantage of "new" features like testing.Run which was added in Go1.7.....)

riannucci avatar Apr 20 '22 18:04 riannucci