LibAFL LibAFL Code Refactoring & Cleanup

We have lots of places where we have type constraints that are not necessary, old code that is not supported/unused, things which we have no tests for, code is disorganised due to the many hands problem, missing documentation, code in the wrong place (e.g., libafl_libfuzzer has a lot of components that should be in libafl_targets), etc. We need to do some spring cleaning for 0.13!

After each file is checked, either a PR is made with the house cleaning items for that file or it is checked off if there is nothing to do. I think if we focus on this one file at a time, we can crack through it in a few days.

If you are not a maintainer and want to do some of these tasks: please open an issue with the list of files/components you want to tackle so we don't duplicate effort.

Specific things to look for:

[ ] Wherever possible, remove #[allow(...)] and apply the clippy fix instead. This was likely done as an easy way out.
[ ] Wherever possible, use Cow<'static, str> instead of String.
[ ] Fix documentation to avoid obvious things, e.g. "this is a constructor", "this is a module". Write usage notes instead.
[ ] Fix documentation where old information is provided.
[ ] #[deprecate = "<reason>"] tags for no-longer-supported things, or things with better alternatives.
[ ] Remove generic restrictions at the definitions (e.g., we do not need to specify that types impl Serialize, Deserialize, or Debug anymore at the struct definitions).
[ ] Reduce generics to the least restrictive necessary.
[ ] If we could (performantly) use safe code instead of unsafe code: rewrite.
[ ] Traits which have an associated type should refer to the associated type, not the concrete/generic.
[ ] PhantomData should have the smallest set of types needed.
[ ] Wherever possible, trait implementations with lifetime specifiers should use '_ lifetime elision.
[ ] Manual Debug impls should be replaced with .debug_struct chains.
[ ] Complex constructors should be replaced with typed_builder.

The following is a checklist of files which need to be cleaned in the main library files:

`libafl`

[ ] libafl/src/corpus/cached.rs
[ ] libafl/src/corpus/inmemory.rs
[ ] libafl/src/corpus/inmemory_ondisk.rs
[ ] libafl/src/corpus/minimizer.rs
[ ] libafl/src/corpus/mod.rs
[ ] libafl/src/corpus/nop.rs
[ ] libafl/src/corpus/ondisk.rs
[ ] libafl/src/corpus/testcase.rs
[ ] libafl/src/events/hooks/mod.rs
[ ] libafl/src/events/centralized.rs
[ ] libafl/src/events/launcher.rs
[ ] libafl/src/events/llmp.rs
[ ] libafl/src/events/mod.rs
[ ] libafl/src/events/simple.rs
[ ] libafl/src/events/tcp.rs
[ ] libafl/src/executors/combined.rs
[ ] libafl/src/executors/command.rs
[ ] libafl/src/executors/hooks/inprocess.rs
[ ] libafl/src/executors/hooks/inprocess_fork.rs
[ ] libafl/src/executors/hooks/mod.rs
[ ] libafl/src/executors/hooks/timer.rs
[ ] libafl/src/executors/hooks/unix.rs
[ ] libafl/src/executors/hooks/windows.rs
[ ] libafl/src/executors/shadow.rs
[ ] libafl/src/executors/with_observers.rs
[ ] libafl/src/executors/inprocess/inner.rs
[ ] libafl/src/executors/inprocess/stateful.rs
[ ] libafl/src/executors/inprocess/mod.rs
[ ] libafl/src/executors/inprocess_fork/inner.rs
[ ] libafl/src/executors/inprocess_fork/mod.rs
[ ] libafl/src/executors/inprocess_fork/stateful.rs
[ ] libafl/src/executors/differential.rs
[ ] libafl/src/executors/forkserver.rs
[ ] libafl/src/executors/mod.rs
[ ] libafl/src/feedbacks/concolic.rs
[ ] libafl/src/feedbacks/differential.rs
[ ] libafl/src/feedbacks/list.rs
[ ] libafl/src/feedbacks/mod.rs
[ ] libafl/src/feedbacks/nautilus.rs
[ ] libafl/src/feedbacks/new_hash_feedback.rs
[ ] libafl/src/feedbacks/stdio.rs
[ ] libafl/src/feedbacks/transferred.rs
[ ] libafl/src/feedbacks/map.rs
[ ] libafl/src/fuzzer/mod.rs
[ ] libafl/src/generators/gramatron.rs
[ ] libafl/src/generators/nautilus.rs
[ ] libafl/src/generators/mod.rs
[ ] libafl/src/inputs/bytes.rs
[ ] libafl/src/inputs/encoded.rs
[ ] libafl/src/inputs/gramatron.rs
[ ] libafl/src/inputs/mod.rs
[ ] libafl/src/inputs/multi.rs
[ ] libafl/src/inputs/nautilus.rs
[ ] libafl/src/inputs/generalized.rs
[ ] libafl/src/monitors/tui/mod.rs
[ ] libafl/src/monitors/tui/ui.rs
[ ] libafl/src/monitors/disk.rs
[ ] libafl/src/monitors/multi.rs
[ ] libafl/src/monitors/prometheus.rs
[ ] libafl/src/monitors/mod.rs
[ ] libafl/src/mutators/string/unicode_categories.rs
[ ] libafl/src/mutators/encoded_mutations.rs
[ ] libafl/src/mutators/gramatron.rs
[ ] libafl/src/mutators/grimoire.rs
[ ] libafl/src/mutators/mod.rs
[ ] libafl/src/mutators/mopt_mutator.rs
[ ] libafl/src/mutators/multi.rs
[ ] libafl/src/mutators/mutations.rs
[ ] libafl/src/mutators/nautilus.rs
[ ] libafl/src/mutators/scheduled.rs
[ ] libafl/src/mutators/string.rs
[ ] libafl/src/mutators/token_mutations.rs
[ ] libafl/src/mutators/tuneable.rs
[ ] libafl/src/observers/concolic/observer.rs
[ ] libafl/src/observers/concolic/metadata.rs
[ ] libafl/src/observers/concolic/mod.rs
[ ] libafl/src/observers/concolic/serialization_format.rs
[ ] libafl/src/observers/stdio.rs
[ ] libafl/src/observers/stacktrace.rs
[ ] libafl/src/observers/value.rs
[ ] libafl/src/observers/cmp.rs
[ ] libafl/src/observers/list.rs
[ ] libafl/src/observers/mod.rs
[ ] libafl/src/observers/map.rs
[ ] libafl/src/schedulers/accounting.rs
[ ] libafl/src/schedulers/minimizer.rs
[ ] libafl/src/schedulers/powersched.rs
[ ] libafl/src/schedulers/probabilistic_sampling.rs
[ ] libafl/src/schedulers/queue.rs
[ ] libafl/src/schedulers/testcase_score.rs
[ ] libafl/src/schedulers/tuneable.rs
[ ] libafl/src/schedulers/weighted.rs
[ ] libafl/src/schedulers/mod.rs
[ ] libafl/src/stages/push/mod.rs
[ ] libafl/src/stages/push/mutational.rs
[ ] libafl/src/stages/logics.rs
[ ] libafl/src/stages/calibrate.rs
[ ] libafl/src/stages/concolic.rs
[ ] libafl/src/stages/dump.rs
[ ] libafl/src/stages/generalization.rs
[ ] libafl/src/stages/mod.rs
[ ] libafl/src/stages/mutational.rs
[ ] libafl/src/stages/power.rs
[ ] libafl/src/stages/stats.rs
[ ] libafl/src/stages/string.rs
[ ] libafl/src/stages/sync.rs
[ ] libafl/src/stages/tracing.rs
[ ] libafl/src/stages/tuneable.rs
[ ] libafl/src/stages/colorization.rs
[ ] libafl/src/stages/tmin.rs
[ ] libafl/src/state/mod.rs
[ ] libafl/src/common/mod.rs
[ ] libafl/src/lib.rs
[ ] libafl/build.rs

`libafl_bolts`

[ ] libafl_bolts/examples/llmp_test/main.rs
[ ] libafl_bolts/src/compress.rs
[ ] libafl_bolts/src/os/pipes.rs
[ ] libafl_bolts/src/os/unix_shmem_server.rs
[ ] libafl_bolts/src/os/unix_signals.rs
[ ] libafl_bolts/src/os/windows_exceptions.rs
[ ] libafl_bolts/src/os/mod.rs
[ ] libafl_bolts/src/build_id.rs
[ ] libafl_bolts/src/staterestore.rs
[ ] libafl_bolts/src/fs.rs
[ ] libafl_bolts/src/math.rs
[ ] libafl_bolts/src/minibsod.rs
[x] #2101
[ ] libafl_bolts/src/anymap.rs
[ ] libafl_bolts/src/cli.rs
[ ] libafl_bolts/src/core_affinity.rs
[ ] libafl_bolts/src/cpu.rs
[ ] libafl_bolts/src/llmp.rs
[ ] libafl_bolts/src/ownedref.rs
[ ] libafl_bolts/src/serdeany.rs
[ ] libafl_bolts/src/shmem.rs
[ ] libafl_bolts/src/lib.rs
[ ] libafl_bolts/src/tuples.rs
[ ] libafl_bolts/build.rs

`libafl_cc`

[ ] libafl_cc/src/ar.rs
[ ] libafl_cc/src/lib.rs
[ ] libafl_cc/src/libtool.rs
[ ] libafl_cc/src/clang.rs
[ ] libafl_cc/src/cfg.rs
[ ] libafl_cc/build.rs

`libafl_concolic`

[ ] libafl_concolic/symcc_libafl/src/lib.rs
[ ] libafl_concolic/symcc_runtime/src/filter/coverage.rs
[ ] libafl_concolic/symcc_runtime/src/filter.rs
[ ] libafl_concolic/symcc_runtime/src/tracing.rs
[ ] libafl_concolic/symcc_runtime/src/lib.rs
[ ] libafl_concolic/symcc_runtime/build.rs
[ ] libafl_concolic/test/dump_constraints/src/main.rs
[ ] libafl_concolic/test/runtime_test/src/lib.rs

`libafl_derive` (does anyone use this?)

[ ] libafl_derive/src/lib.rs

`libafl_frida`

[ ] libafl_frida/src/asan/mod.rs
[ ] libafl_frida/src/asan/asan_rt.rs
[ ] libafl_frida/src/asan/errors.rs
[ ] libafl_frida/src/asan/hook_funcs.rs
[ ] libafl_frida/src/windows_hooks.rs
[ ] libafl_frida/src/drcov_rt.rs
[ ] libafl_frida/src/coverage_rt.rs
[ ] libafl_frida/src/alloc.rs
[ ] libafl_frida/src/cmplog_rt.rs
[ ] libafl_frida/src/executor.rs
[ ] libafl_frida/src/helper.rs
[ ] libafl_frida/src/lib.rs
[ ] libafl_frida/src/pthread_hook.rs
[ ] libafl_frida/src/utils.rs
[ ] libafl_frida/build.rs

`libafl_libfuzzer`

[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/tmin.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/corpus.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/feedbacks.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/fuzz.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/lib.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/merge.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/misc.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/options.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/report.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/schedulers.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/src/observers.rs
[ ] libafl_libfuzzer/libafl_libfuzzer_runtime/build.rs
[ ] libafl_libfuzzer/src/lib.rs
[ ] libafl_libfuzzer/build.rs

`libafl_nyx`

[ ] libafl_nyx/build.rs
[ ] libafl_nyx/src/executor.rs
[ ] libafl_nyx/src/helper.rs
[ ] libafl_nyx/src/lib.rs
[ ] libafl_nyx/src/settings.rs

`libafl_qemu`

[ ] libafl_qemu/libafl_qemu_build/src/main.rs
[ ] libafl_qemu/libafl_qemu_build/src/bindings.rs
[ ] libafl_qemu/libafl_qemu_build/src/build.rs
[ ] libafl_qemu/libafl_qemu_build/src/lib.rs
[ ] libafl_qemu/libafl_qemu_sys/build.rs
[ ] libafl_qemu/libafl_qemu_sys/src/lib.rs
[ ] libafl_qemu/libafl_qemu_sys/src/systemmode.rs
[ ] libafl_qemu/libafl_qemu_sys/src/usermode.rs
[ ] libafl_qemu/libafl_qemu_sys/src/x86_64_stub_bindings.rs
[ ] libafl_qemu/libafl_qemu_sys/build_linux.rs
[ ] libafl_qemu/src/executor/mod.rs
[ ] libafl_qemu/src/executor/stateful.rs
[ ] libafl_qemu/src/aarch64.rs
[ ] libafl_qemu/src/arm.rs
[ ] libafl_qemu/src/asan.rs
[ ] libafl_qemu/src/asan_guest.rs
[ ] libafl_qemu/src/breakpoint.rs
[ ] libafl_qemu/src/calls.rs
[ ] libafl_qemu/src/cmplog.rs
[ ] libafl_qemu/src/command.rs
[ ] libafl_qemu/src/drcov.rs
[ ] libafl_qemu/src/elf.rs
[ ] libafl_qemu/src/emu.rs
[ ] libafl_qemu/src/emu/systemmode.rs
[ ] libafl_qemu/src/emu/usermode.rs
[ ] libafl_qemu/src/helper.rs
[ ] libafl_qemu/src/hexagon.rs
[ ] libafl_qemu/src/hooks.rs
[ ] libafl_qemu/src/i386.rs
[ ] libafl_qemu/src/injections.rs
[ ] libafl_qemu/src/lib.rs
[ ] libafl_qemu/src/mips.rs
[ ] libafl_qemu/src/ppc.rs
[ ] libafl_qemu/src/snapshot.rs
[ ] libafl_qemu/src/sync_backdoor.rs
[ ] libafl_qemu/src/x86_64.rs
[ ] libafl_qemu/src/edges.rs
[ ] libafl_qemu/build.rs
[ ] libafl_qemu/build_linux.rs

`libafl_sugar`

[ ] libafl_sugar/build.rs
[ ] libafl_sugar/src/lib.rs
[ ] libafl_sugar/src/inmemory.rs
[ ] libafl_sugar/src/qemu.rs
[ ] libafl_sugar/src/forkserver.rs

`libafl_targets`

[ ] libafl_targets/src/forkserver.rs
[ ] libafl_targets/src/value_profile.rs
[ ] libafl_targets/src/drcov.rs
[ ] libafl_targets/src/libfuzzer/observers/mod.rs
[ ] libafl_targets/src/libfuzzer/observers/oom.rs
[ ] libafl_targets/src/libfuzzer/mod.rs
[ ] libafl_targets/src/libfuzzer/mutators.rs
[ ] libafl_targets/src/cmps/observers/mod.rs
[ ] libafl_targets/src/cmps/observers/aflpp.rs
[ ] libafl_targets/src/cmps/observers/cmplog.rs
[ ] libafl_targets/src/cmps/stages/mod.rs
[ ] libafl_targets/src/cmps/stages/aflpptracing.rs
[ ] libafl_targets/src/cmps/mod.rs
[ ] libafl_targets/src/windows_asan.rs
[ ] libafl_targets/src/lib.rs
[ ] libafl_targets/src/sancov_cmp.rs
[ ] libafl_targets/src/sancov_8bit.rs
[ ] libafl_targets/src/coverage.rs
[ ] libafl_targets/src/sancov_pcguard.rs
[ ] libafl_targets/build.rs

`libafl_tinyinst`

[ ] libafl_tinyinst/src/lib.rs
[ ] libafl_tinyinst/src/executor.rs

Apr 17 '24 17:04 addisoncrump

I would say this is a good step towards 1.0

Apr 17 '24 20:04 domenukk

I want to take on libafl_bolts/rands.rs. I've already submitted a couple of improvement PRs there, and would like to finish the job.

Apr 25 '24 06:04 flyingmutant

One thing to add; We should use builder pattern for the constructor as much as possible. And a few modules need this.

Apr 30 '24 12:04 tokatoka

typed_builder usually makes this very comfortable

Apr 30 '24 13:04 domenukk

LibAFL LibAFL copied to clipboard

LibAFL Code Refactoring & Cleanup

LibAFL
LibAFL copied to clipboard