elements icon indicating copy to clipboard operation
elements copied to clipboard

simplicity: add fuzz target

Open delta1 opened this issue 1 year ago • 12 comments

adds a new fuzz target for elements_simplicity_execSimplicity

compile with instructions in fuzzing.md

run with: FUZZ=simplicity src/test/fuzz/fuzz

delta1 avatar May 06 '24 13:05 delta1

as this fuzz target is now, quickly found these 2 crashes and a leak fuzz.zip

delta1 avatar May 06 '24 13:05 delta1

Do you know how to use these inputs?

apoelstra avatar May 06 '24 18:05 apoelstra

Unfortunately I can't test this because I cannot figure out how to get a nix-shell in which the fuzz tests are built in the same way as nix-build does. Somehow stepping through the unpackPhase/autoreconfPhase/configurePhase/buildPhase in a nix-shell --pure results in linker errors.

apoelstra avatar May 07 '24 01:05 apoelstra

Do you know how to use these inputs?

The input can be passed as an arg

> FUZZ=simplicity src/test/fuzz/fuzz ./crash-631f08c5ca034c7a8cb316a45ec1aaf855360cc9
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3819231386
INFO: Loaded 1 modules   (247138 inline 8-bit counters): 247138 [0x5c6c425dbdf3, 0x5c6c42618355), 
INFO: Loaded 1 PC tables (247138 PCs): 247138 [0x5c6c42618358,0x5c6c429dd978), 
src/test/fuzz/fuzz: Running 1 inputs 1 time(s) each.
Running: ./crash-631f08c5ca034c7a8cb316a45ec1aaf855360cc9
AddressSanitizer:DEADLYSIGNAL
=================================================================
==2179002==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x5c6c415b43b9 bp 0x000000000000 sp 0x7ffd9e2fec50 T0)
==2179002==The signal is caused by a READ memory access.
==2179002==Hint: address points to the zero page.
    #0 0x5c6c415b43b9 in ReadBE32 /home/byron/code/elements/src/simplicity/primitive/elements/../../sha256.h:38:8
    #1 0x5c6c415b43b9 in sha256_toMidstate /home/byron/code/elements/src/simplicity/primitive/elements/../../sha256.h:100:17
    #2 0x5c6c415b43b9 in elements_simplicity_mallocTapEnv /home/byron/code/elements/src/simplicity/primitive/elements/env.c:553:3
    #3 0x5c6c405a5e92 in simplicity_fuzz_target(Span<unsigned char const>) ./test/fuzz/simplicity.cpp:42:31

Unfortunately I can't test this because I cannot figure out how to get a nix-shell in which the fuzz tests are built in the same way as nix-build does. Somehow stepping through the unpackPhase/autoreconfPhase/configurePhase/buildPhase in a nix-shell --pure results in linker errors.

I tried to get a shell.nix working with clangstdenv but couldn't get it to compile

delta1 avatar May 07 '24 10:05 delta1

The other crash is in the same place as above, I'll look into it.

The "leak" was just because I didn't free, added that now.

delta1 avatar May 07 '24 10:05 delta1

Updated to include the expected invariants for control block and script_bytes.

Haven't found any crashes now, and submitted some seed data in https://github.com/ElementsProject/qa-assets/pull/2

Run with: FUZZ=simplicity src/test/fuzz/fuzz /path/to/qa-assets/fuzz_seed_corpus/simplicity

delta1 avatar May 08 '24 11:05 delta1

One thing we should verify is that the PRODUCTION flag is not enable during fuzzing to ensure all assertions are being run:

https://github.com/BlockstreamResearch/simplicity/blob/master/C/simplicity_assert.h#L12

If there is some sort of FUZZ define, we can add a check/warning to the simplicity code that FUZZ and PRODUCTION are not both set.

roconnor-blockstream avatar May 08 '24 15:05 roconnor-blockstream

@delta1 I don't know why, but running the command

$ PRINT_ALL_FUZZ_TARGETS_AND_ABORT= src/test/fuzz/fuzz

doesn't list "simplicity" for some reason.

roconnor-blockstream avatar May 25 '24 17:05 roconnor-blockstream

@roconnor-blockstream I saw the same issue, made a change in fuzz.cpp and recompiled, and now it seems to appear? Maybe ccache related, are you using that too?

Here's my output with some lines removed for clarity

> PRINT_ALL_FUZZ_TARGETS_AND_ABORT= src/test/fuzz/fuzz
addition_overflow
...
signet
simplicity
snapshotmetadata_deserialize
...
witness_program
fuzz: test/fuzz/fuzz.cpp:66: auto initialize()::(anonymous class)::operator()() const: Assertion `"!should_abort" && check' failed.
zsh: IOT instruction (core dumped)  PRINT_ALL_FUZZ_TARGETS_AND_ABORT= src/test/fuzz/fuzz

The change I made before recompiling was:

diff --git a/src/test/fuzz/fuzz.cpp b/src/test/fuzz/fuzz.cpp
index a33297e0ed..800bcd0354 100644
--- a/src/test/fuzz/fuzz.cpp
+++ b/src/test/fuzz/fuzz.cpp
@@ -27,6 +27,7 @@ std::map<std::string_view, std::tuple<TypeTestOneInput, TypeInitialize, TypeHidd
 
 void FuzzFrameworkRegisterTarget(std::string_view name, TypeTestOneInput target, TypeInitialize init, TypeHidden hidden)
 {
+    std::cout << "register: " << name << std::endl;
     const auto it_ins = FuzzTargets().try_emplace(name, std::move(target), std::move(init), hidden);
     Assert(it_ins.second);
 }

Which I then confirmed printed register: simplicity and then saw simplicity in the target list...

delta1 avatar May 27 '24 09:05 delta1

Oops I was building #1326 instead of this PR. My bad.

roconnor-blockstream avatar May 27 '24 17:05 roconnor-blockstream

Please rebase this PR. Thanks.

roconnor-blockstream avatar Jun 06 '24 15:06 roconnor-blockstream

@roconnor-blockstream rebased

delta1 avatar Jun 07 '24 07:06 delta1