Add advanced fuzzer

[maxammann]

Adds a fuzzer which:

• Can call contracts
• Uses complete scripts with fuzzed script data
• Can use LibAFL

Manual

cargo install cargo-fuzz
apt install clang pkg-config libssl-dev # for LibAFL
rustup component add llvm-tools-preview --toolchain nightly

Generate Seeds

It is necessary to first convert Sway programs into a suitable format for use as seed input to the fuzzer. This can be done with the following command:

cargo run --bin seed <input dir> <output dir>

Running the Fuzzer

The Rust nightly version is required for executing cargo-fuzz. We also disable AddressSanitizer for a significant speed improvement, as we do not expect memory issues in a Rust program that does not use a significant amount of unsafe code, which our cargo-geiger analysis showed. It makes sense to leave AddressSanitizer turned on if the Fuel project uses more unsafe Rust in the future (either directly or through dependencies). The remaining flags are either required for LibAFL or are useful to make it use seven cores.

cargo +nightly fuzz run --sanitizer none grammar_aware -- \
	-ignore_crashes=1 -ignore_timeouts=1 -ignore_ooms=1 -fork=7

If you use libfuzzer (default) then the following command is enough:

cargo fuzz run --sanitizer none grammar_aware

Execute a Test Case

Test cases can be executed using the following command. This is useful for triaging issues.

cd fuzz/
cargo run --bin execute <file/dir>

Collect Statistics

We created a tool that writes gas statistics to a file called gas_statistics.csv. This can be used to analyze the execution time versus gas usage on a test corpus.

cargo run --bin collect <file/dir>

Generate Coverage

Regardless of how inputs are generated, it is important to measure a fuzzing campaign’s coverage after its run. To perform this measure, we used the support provided by cargo-fuzz and rustc. First, install cargo-binutils. After that, execute the following command:

cargo +nightly fuzz coverage grammar_aware corpus/grammar_aware

Finally, generate an HTML file using LLVM:

cargo cov -- show
target/x86_64-unknown-linux-gnu/coverage/x86_64-unknown-linux-gnu/release/grammar_aware --format=html -instr-profile=coverage/grammar_aware/coverage.profdata /root/audit/fuel-vm > index.html

The last argument to cargo cov defines the filter based on the host file paths.

Checklist

• [x] Breaking changes are clearly marked as such in the PR description and changelog
• [x] New behavior is reflected in tests
• [x] If performance characteristic of an instruction change, update gas costs as well or make a follow-up PR for that
• [x] The specification matches the implemented behavior (link update PR if changes are needed)

Before requesting review

• [x] I have reviewed the code myself
• [x] I have created follow-up issues caused by this PR and linked them here

After merging, notify other teams

TODO

• [ ] Rust SDK
• [ ] Sway compiler
• [ ] Platform documentation (for out-of-organization contributors, the person merging the PR will do this)
• [ ] Someone else?

[CLAassistant]

All committers have signed the CLA.

[maxammann]

@xgreenx Let me know how you want the corpus to be delivered :) I suggest an external repository that can be public or private. Private might be better to start with.

I wanted to experiment with integrating a CI fuzzing, but that can follow up as separate PR.

[maxammann]

Just an overall question: do you have any ideas on how it can be integrated into our CI on a daily basis?

My idea was to experiment with https://google.github.io/clusterfuzzlite/ to add CI fuzzing in a future PR.

[maxammann]

@xgreenx Here is the CFL PR: https://github.com/FuelLabs/fuel-vm/pull/727

[maxammann]

@xgreenx I think we want to add the no-changelog tag to this PR. Also the wasm checks seem to depend on the base branch being in the fuel-vm repository.