Integrate with fuzzcheck?

[loiclec]

Hello 👋

I am the maintainer of fuzzcheck, which is a fuzzing engine built for Rust. For now, fuzzcheck works in the same way as cargo-fuzz in that one needs to create a fuzz folder, and then a fuzz target per test function. It is a bit cumbersome and I'd like to find a better solution. I really like the idea of auto-fuzz test, and I'd like to have something similar that works with fuzzcheck.

I am wondering whether it would be best to either:

1. add support for fuzzcheck to auto-fuzz-test; or
2. write similar functionality directly into fuzzcheck

I don't mind at all doing (2) by myself, but I thought it may be rude to do so without contacting you in the first place, since you may want (1) too, and in that case we should collaborate on it :)

There are a few important differences between fuzzcheck and cargo-fuzz that would make its integration a bit more difficult though. Whereas the fuzz targets of cargo-fuzz look like this:

// fuzz/fuzz_targets/target1.rs

libfuzzer_sys::fuzz_target!(|color: Rgb| {
     my_library::test(color)
});

The fuzz targets of fuzzcheck need to do a bit of setup to choose an appropriate Mutator and Serializer:

// fuzz/non_instrumented/fuzz_targets/target1.rs
// many `use` statements...
fn main() {
    let mutator = Rgb::default_mutator();
    let serializer = SerdeSerializer::default();
    let _ = fuzzcheck::launch(my_library::test, mutator, serializer);
}

While choosing default_mutator() and SerdeSerializer should be good for many cases, there should probably be an option to specify these two somewhere.

Another big difference between the two is that the arguments to a test function in fuzzcheck cannot be modified, even internally. So neither &mut T nor Cell<T> are allowed. If we have a function like:

fn foo(x: &mut Rgb) {
    // ...
}

Then the fuzz target should be:

fn test_foo(x: &Rgb) {
    let mut x = x.clone();
    foo(&mut x);
}

There may be other problems, but these are the biggest ones I can think of right now.

Let me know if you'd like to work together to add support for fuzzcheck to auto-fuzz-test, or whether you would rather prefer to support only cargo-fuzz. But again, I really don't want to put any pressure on you to do anything :)

[totikom]

I think, I can try to implement fuzzcheck support into auto-fuzz.

The fuzz targets of fuzzcheck need to do a bit of setup to choose an appropriate Mutator and Serializer

This doesn't seems difficult, as I only have to change the fuzz_targets generation functions. Probably, the only thing I really have to change is the folder creation logic and fuzzing harness template.

Immutability of the arguments is a bit harder, but that's ok. As a MVP, I can just clone the whole input and use it.)

It's probably going to take me a long time to implement this, as now I am working on my bachelor's thesis.

[Shnatsel]

Hi Loïc!

auto-fuzz-test is in the exploratory stage right now. We're happy to accept PRs adding fuzzcheck support! We have not particularly invested in any single approach, so breaking changes are fine too!

I feel making a chat room would help collaboration tremendously, since auto-fuzz-test has been rapidly evolving and is not extensively documented. @totikom @loiclec do you have any preferred chat platforms? How do you feel about Discord? I see a lot of Rust projects use it, and it seems to work fine.

[loiclec]

I am really glad to hear you're both open to the idea!

I agree a chat room would be nice. I haven't used any chat platforms except for Slack a few years ago, so I really don't mind. Discord sounds good too!

[Shnatsel]

Cool, I've created a Discord server: https://discord.gg/XWfqMFZh

[totikom]

Discord sounds fine. Martrix is also good.