LibAFL support?

[domenukk]

Now that libfuzzer has been officially deprecated I was wondering if the maintainers of cargo-fuzz are interested to add LibAFL support to this crate directly.

LibAFL is not only maintained, but is written in Rust and has overall better performance in benchmarks.

So far, we maintain a fork of cargo-fuzz, called cargo-libafl and LibAFL can be a drop-in replacement for libfuzzer in general.

We can help out integrating this, but would need to know how to best add support, if this is something you are interested in.

[fitzgen]

I'm open to accepting pull requests that implement this. But I think we need design work before that happens.

One thing to figure out before hand is what the developer experience will be like and how this will/won't affect the CLI flags/commands. Ideally we won't require passing --libafl or --libfuzzer to every single command. Perhaps we can configure this in Cargo.toml?

[Manishearth]

FWIW Given that libFuzzer is not deprecated but in maintenance mode i'd prefer to make other fuzzers non-default options, fwiw.

Configuration in Cargo.toml makes sense, we can set the configuration on cargo fuzz init and init can have flags

[fitzgen]

Agreed.

[addisoncrump]

This is precisely why I opened https://github.com/rust-fuzz/libfuzzer/pull/103 :grin: (liblibafl...)

I'm finding what libafl lacks for compatibility and working on integration there: https://github.com/AFLplusplus/LibAFL/tree/libfuzzer As of now, there are a couple of sancov items used that aren't currently supported by libafl, but most of the libfuzzer features are already supported. Should be a fairly straightforward replacement after that.

I'll bump here once we're compatible with CUSTOM_LIBFUZZER_PATH; after that, it should be fairly trivial to integrate via libfuzzer-sys.

[frewsxcv]

Considering changing the underlying fuzzer can be a breaking change (because of libfuzzer specific flags?), I think it's also worth having a discussion of how we introduce breaking changes in cargo-fuzz going forward.

[addisoncrump]

Actually, in this case, no -- the libfuzzer shim we wrote supports the same flags as libfuzzer, with some exceptions that won't affect the common use case. I already use my shim with cargo-fuzz for testing purposes.

[addisoncrump]

It also generally outperforms libfuzzer, but we're still working on some additional improvements (namely, some compatibility issues with macOS and Windows)

[addisoncrump]

This is now supported by libfuzzer-sys via https://github.com/rust-fuzz/libfuzzer/pull/106.

[addisoncrump]

We have now implemented compatibility with cargo-fuzz by upstreaming changes to libfuzzer-sys: https://github.com/rust-fuzz/libfuzzer/pull/106

It is now possible to switch to our fuzzer by replacing the libfuzzer-sys dependency:

#libfuzzer-sys = { version = "*", features = ["arbitrary-derive"] }
libfuzzer-sys = { git = "https://github.com/AFLplusplus/LibAFL.git", features = ["arbitrary-derive"], package = "libafl_libfuzzer" }

We are still fleshing out these changes, but our initial results show promising performance improvements (time to bug, time to coverage) over libfuzzer :grin:

[wcampbell0x2a]

@addisoncrump if I use -j for cargo fuzz do the fuzzers talk to each other like normal libAFL?

[wcampbell0x2a]

Also! How do I get stats on the amount of current coverage while it's running?

[addisoncrump]

Hey @wcampbell0x2a; let's discuss further on the LibAFL issue for this: https://github.com/AFLplusplus/LibAFL/pull/981

Yes, you can use -j as normal and the current coverage stats should be output as it runs, though some users are experiencing inconsistency in the output (there's a reason this isn't merged just yet!).

[Dr-Emann]

It looks like the upstream AFL++ change has been merged, what else needs to happen for this?

[addisoncrump]

This is definitely completed, I just can't close the issue.

[domenukk]

cargo fuzz init doesn't have a flag yet to make this easy, does it?

[fitzgen]

Going to close this issue as the initial support has landed. Feel free to open new issues and make new PRs for further improvements.