foundry icon indicating copy to clipboard operation
foundry copied to clipboard

meta(fuzzer): tracking issue for fuzzer improvements

Open gakonst opened this issue 1 year ago • 3 comments

https://twitter.com/gakonst/status/1790770389523583163

Fuzz tests

Fuzzing-testing specific improvements, continuation of https://github.com/foundry-rs/foundry/issues/4433 and https://github.com/foundry-rs/foundry/pull/10190

UX/Features

High

  • https://github.com/foundry-rs/foundry/issues/4300
  • https://github.com/foundry-rs/foundry/issues/990
  • https://github.com/foundry-rs/foundry/issues/8788
  • https://github.com/foundry-rs/foundry/issues/10877: stateless fuzzing support and additional ABI mutations

Nice to have

  • https://github.com/foundry-rs/foundry/issues/4271
  • https://github.com/foundry-rs/foundry/issues/8117
  • https://github.com/foundry-rs/foundry/issues/4097
  • https://github.com/foundry-rs/foundry/issues/2552 - change existing format to standard when available, see https://github.com/crytic/medusa/issues/234
  • https://github.com/foundry-rs/foundry/issues/3844
  • https://github.com/foundry-rs/foundry/issues/12332

Bugs

  • https://github.com/foundry-rs/foundry/issues/6529
  • https://github.com/foundry-rs/foundry/issues/6623
  • https://github.com/foundry-rs/foundry/issues/8639
  • https://github.com/foundry-rs/foundry/issues/12330
  • https://github.com/foundry-rs/foundry/issues/12331

Invariants

Invariant-testing specific improvements, continuation of https://github.com/foundry-rs/foundry/issues/4438

UX/Features

High

  • https://github.com/foundry-rs/foundry/issues/3607
  • https://github.com/foundry-rs/foundry/issues/4007
  • https://github.com/foundry-rs/foundry/issues/8898: share corpus and run as many invariants in as many threads as possible
  • https://github.com/foundry-rs/foundry/issues/9727: ignore crashes to allow continuous fuzzing
  • https://github.com/foundry-rs/foundry/pull/8644: fuzz msg.value
  • https://github.com/foundry-rs/foundry/issues/10875: seed corpus from tests
  • https://github.com/foundry-rs/foundry/issues/10233: insert constants and evaluated constant expressions in source in to fuzzer dictionary
  • https://github.com/foundry-rs/foundry/pull/10190#discussion_r2114069107 (Maybe no longer needed: optimize the data structure of the corpus for lookups)
  • [ ] implement compile-time, non-colliding instrumentation like afl++ PCGUARD in Solar and coverage-guided fuzzing to use it
  • https://github.com/foundry-rs/foundry/issues/12190
  • https://github.com/foundry-rs/foundry/issues/12268
  • https://github.com/foundry-rs/foundry/issues/12269

Nice to have

  • https://github.com/foundry-rs/foundry/issues/5018
  • https://github.com/foundry-rs/foundry/issues/8114
  • https://github.com/foundry-rs/foundry/issues/2985
  • https://github.com/foundry-rs/foundry/issues/4352
  • https://github.com/foundry-rs/foundry/issues/4091
  • https://github.com/foundry-rs/foundry/issues/3452
  • https://github.com/foundry-rs/foundry/issues/1578
  • https://github.com/foundry-rs/foundry/issues/10876: campaign stats logging
  • https://github.com/foundry-rs/foundry/issues/12333
    • [ ] add gas/s
  • https://github.com/foundry-rs/foundry/issues/12526

Performance

High

  • https://github.com/foundry-rs/foundry/issues/8116
  • https://github.com/foundry-rs/foundry/issues/8115
  • https://github.com/foundry-rs/foundry/issues/4287
  • https://github.com/foundry-rs/foundry/issues/4163
  • https://github.com/foundry-rs/foundry/issues/2986

Benchmarks

High

  • https://github.com/foundry-rs/foundry/issues/3411
    • set up daily runner of https://github.com/grandizzy/fuzz-benchmarks/ + add more tests
  • see details in https://github.com/foundry-rs/foundry/issues/7610
    • run https://github.com/foundry-rs/foundry/issues/4590
    • report as suggested in https://github.com/fuzz-evaluator/guidelines

Symbolic execution

  • https://github.com/foundry-rs/foundry/issues/15

Mutation testing

  • https://github.com/foundry-rs/foundry/issues/478

gakonst avatar Jun 06 '24 00:06 gakonst

this would be neat: https://github.com/foundry-rs/foundry/issues/4000

zerosnacks avatar Jun 27 '24 13:06 zerosnacks

this would be neat: #4000

yep, indeed! made a PR for it here https://github.com/foundry-rs/foundry/pull/8338

grandizzy avatar Jul 03 '24 09:07 grandizzy