LibAFL Improving Handling of Custom Inputs

I've finally gotten around to doing something about #2202. It's currently still very much work in progress.

Still missing:

[x] Optional Input parts and crossover mutations
- [x] Ideally, I'd find a way to extract the mapping logic from the actual mutators, so that it can be reused in the crossover mutators as well
- [x] This would also possibly require a reversal of ownership of the mapping logic within mutators, as the logic would need to be called inside-out instead of outside-in
[ ] Possibly additional mappers, I'm open to ideas
[x] (Improved) Documentation, including examples
[x] Tests on all introduced parts
[ ] Not sure if this is possible, but it would be cool if the names of the mapping mutators included the names of the inner mutator — I've failed at wrangling the lifetime checker to a working implementation so far
[ ] Not sure if this is possible either, but mapping on tuple_list_type would drastically reduce code duplication

Feel free to leave any suggestions or questions!

Jul 17 '24 20:07 riesentoaster

Besides the optional improvements outlined above, I think this is done.

Aug 07 '24 13:08 riesentoaster

One other thing: It'd be pretty amazing to #[derive] mappings for each sub part of an input (maybe even recursiveley), so we suddenly end up with a cheap grammar fuzzer

Aug 08 '24 10:08 domenukk

One other thing: It'd be pretty amazing to #[derive] mappings for each sub part of an input (maybe even recursiveley), so we suddenly end up with a cheap grammar fuzzer

That's the end goal 😎.

One step at a time though.

Aug 08 '24 10:08 riesentoaster

@riesentoaster please take a look at c443dbe939ba75d881b6ed9831fc031106019cb6 of #2479

This is how I would like it. The custom input now needs to implement:

    pub fn mutate_byte_array<M: for<'a> Mutator<MutVecInput<'a>, S>, S>(
        &mut self,
        state: &mut S,
        mutator: &mut M,
    ) -> Result<MutationResult, Error> {
        let mut byte_array = MutVecInput::from(&mut self.byte_array);
        mutator.mutate(state, &mut byte_array)
    }

It's more code than returning a Vec, but this can easily be auto-generated with a macro. And it will also work for any non-vec, non-bytes, and higher-level inputs.

Please take a look/comment :)

/edit: (Especially given the end goal to derive these, the extra code should not make any difference)

Aug 09 '24 10:08 domenukk

I see — thank you for your suggestion!

I think I personally would prefer the simplicity of my initial version for most inputs. If you look at the file havoc_mutations.rs, you see that the most complicated thing about this are the types, the actual logic in this file is fairly straightforward. So it'd be easy to create another additional function that returns mapped havoc_mutations with your interface, if having this level of flexibility is actually needed regularly.

Alternatively, it's fairly straightforward to just create the mappings yourself in your fuzzer — that's the entire point of this architecture: you can plug your mappings together (including creating custom mappings for any custom type) as you like. An example of this is what I did with Option wrapped parts. So the flexibility is there regardless of the simplified interface we provide in havoc_mutations.rs.

Overall, I think it's just a question of taste. And I prefer the version where I don't need to expect that the users remember to call .mutate() manually to get it to work. 😄

Aug 10 '24 17:08 riesentoaster

Sorry for the late response, I was on holidays:

I think I personally would prefer the simplicity of my initial version for most inputs.

The issue is that your version only works with Vec<u8> which is very inflexible.

An example of this is what I did with Option wrapped parts.

This example is a good example: it needs a full range of optional_mapped_havoc_mutations copy & pasted. Any new use case will need more copy & paste. In general we try to get around as much code deduplication as possible.

Maybe we can come up with a flexible middle ground? For example, we could return BytesSubInput instead of Vec<u8>, or similar?

Sep 04 '24 14:09 domenukk

No worries – I hope you enjoyed your time off!

The issue is that your version only works with Vec which is very inflexible.

Fair enough, this is the tradeoff I chose. If you need something based on say a MutVecInput, you can (in just 3 lines!) simply create the mapping yourself in your fuzzer — and you don't even need to write out the types:

havoc_mutations_no_crossover()
    .merge(havoc_crossover_with_corpus_mapper(input_from_corpus_mapper))
    .map(ToFunctionMappingMutatorMapper::new(current_input_mapper))

If it weren't for the need for explicit (and repetitive) types, I would have added another interface for this.

Optionals:

Yes, the types for optional_mapped_havoc_mutations are very similar to those of mapped_havoc_mutations, and I really dislike this as well, hence my idea for mapping for tuple_list_type in the initial message of this PR.
I added this, not just because I want to use it for my project, but also because a user can use this to prevent the mutator from mutating/pulling in data from another input value with any custom logic, based on any (including multiple) field of the input. Think of it as disabling a specific set of mutators that mutate a sub-part of the input, because they may not apply.
And it's another example of how these mappers are easily combinable.
I think I'd be happy to drop initial support for these in the library but instead construct them myself in the fuzzer — however I do need support for an optional mapping for the crossover mutators.

Sep 05 '24 13:09 riesentoaster

Also: I attempted to merge main and saw that for some reason, StdScheduledMutator will no longer accept a tuple_list of mutators that got appended. Haven't had time to investigate, might write an issue.

Side note: One significant downside of the tuple_list structures are these kinds of error messages, and at least to me, these are basically unusable:

error[E0277]: the trait bound `((libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::byte_array_optional}>>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::byte_array_optional}>>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::optional_byte_array_optional}>>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::optional_byte_array_optional}>>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, ())))))))))))))))))))))))))))))))))))))))))))))))))))))), input::ToggleOptionalByteArrayMutator): libafl::mutators::Mutator<_ _>` is not satisfied --> src/main.rs:147:19 | 147 | let mutator = StdScheduledMutator::new(mutations); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `libafl::mutators::Mutator<_ _>` is not implemented for `((FunctionMappingMutator, ...>, ...), ...)`, which is required by `(((libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::byte_array_optional}>>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::mutvecmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::byte_array_optional}>>, for fn(&'a mut input::CustomInput) -> &'a mut std::vec::Vec {input::CustomInput::byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::optional_byte_array_optional}>>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, (libafl::mutators::FunctionMappingMutator<:mutators::optionmappingmutator for> fn(&'a input::CustomInput) -> std::option::Option {input::CustomInput::optional_byte_array_optional}>>>, for fn(&'a mut input::CustomInput) -> &'a mut std::option::Option<:vec::vec>> {input::CustomInput::optional_byte_array_mut}>, ())))))))))))))))))))))))))))))))))))))))))))))))))))))), input::ToggleOptionalByteArrayMutator), input::ToggleBooleanMutator): libafl::mutators::MutatorsTuple<_ _>` | = help: the following other types implement trait `libafl::mutators::Mutator`: <:togglebooleanmutator as libafl::mutators::mutator s>> <:toggleoptionalbytearraymutator as libafl::mutators::mutator s>> <:mutators::bitflipmutator as libafl::mutators::mutator s>> <:mutators::byteaddmutator as libafl::mutators::mutator s>> <:mutators::bytedecmutator as libafl::mutators::mutator s>> <:mutators::byteflipmutator as libafl::mutators::mutator s>> <:mutators::byteincmutator as libafl::mutators::mutator s>> <:mutators::byteinterestingmutator as libafl::mutators::mutator s>> and 48 others = note: required for `(((FunctionMappingMutator, ...>, ...), ...), ...)` to implement `libafl::mutators::MutatorsTuple<_ _>` note: required by a bound in `libafl::mutators::StdScheduledMutator` --> /home/ubuntu/LibAFL/libafl/src/mutators/scheduled.rs:104:9 | 102 | pub struct StdScheduledMutator | ------------------- required by a bound in this struct 103 | where 104 | MT: MutatorsTuple, | ^^^^^^^^^^^^^^^^^^^ required by this bound in `StdScheduledMutator` = note: the full name for the type has been written to '/home/ubuntu/LibAFL/fuzzers/baby/baby_fuzzer_custom_input/target/debug/deps/baby_fuzzer_custom_input-1da2d3a5992f8695.long-type-14408451013759534413.txt' = note: consider using `--verbose` to print the full type name to the console

Sep 05 '24 13:09 riesentoaster

Fair enough, this is the tradeoff I chose. If you need something based on say a MutVecInput, you can (in just 3 lines!) simply create the mapping yourself in your fuzzer

Taking a step back, there are basically two approaches:

a) Call an input mapper function with the original Mutator and the input to be mapped (see my PR). b) Have a mapper function that maps from type a to type b, type b must be compatible to the original Mutator (this PR).

A is very flexible: the wrapper works for anything that's compatible with the given trait (and you can even chain mutliple mappers) I'm happy with b) if it's as flexible as possible: I went down this path originally but had issues with the borrow checker I couldn't overcome, since returning the mapped input that needs to borrow the original input is quite hard(!). If you can make a more generic b) (I don't want a Vec in there ;) ), I'd be happy to go with b. Right now the solution is too specific for my taste, to go into a library.

If it's too hard to pull this off with traits and generics, we could use the concrete BytesSubInput type as suggested above. Maybe that way it'd work. In the "worst" case there's a tiny overhead for bounds checks for the sub input though, but maybe that's fine.

What do you think?

Sep 05 '24 21:09 domenukk

I'm not quite sure I understand where you'd need flexibility that isn't available with the system in this PR. Could you provide an example? Maybe one where you struggled with the borrow checker?

Since FunctionMappingMutator has no requirements on either the outer or inner types, you should be able to use it to bridge between basically anything. This should not only work for havoc style mutators, but any kind incl. AST based inputs etc. So the mapping infrastructure is about as flexible as possible I think.

MutVecMappingMutator and OptionMappingMutator are essentially just special cases of FunctionMappingMutator with a fixed mapping function. I added them to (a) show how the pattern can be used for custom mappings and (b) to use for an interface to havoc_mutations. The way I would approach having custom inputs where havoc style mutations would apply is by using Vec<u8>, just like in the example fuzzer, hence the decision to use that in the interface for mapped_havoc_mutations and optional_mapped_havoc_mutations. But those are essentially just sugar around the simple stacks of custom mapping mutators in havoc_mutations.rs, which in turn are just sugar around the most generic FunctionMappingMutator.

Sep 06 '24 08:09 riesentoaster

If you look at my PR, you can do this:

    let mutations = mapped_havoc_mutations(
        CustomInput::mutate_byte_array
    )

The one list of mapped_havoc_mutations takes a mapper function. This will work for any custom input type, as long as it can map to something that implements HasMutatorBytes after the mutate_byte_array transformation.

Instead of mutate_byte_array, we could instead have a mapping function (like map_byte_array) that returns the mapped Input type, and then we call the mutator outside of that function. However I failed to do this in a borrow-checker friendly way. Does that roughly make sense?

Sep 06 '24 21:09 domenukk

I think I understood what you mean. I added MutVecFunctionMappingMutator, where a user needs to provide a mapping function with type for<'a> FnMut(&'a mut IO) -> MutVecInput<'a> for your custom input type IO.

I also added another field in the custom input of the example, this is all that needed to change in the example:

Mapper functions for the custom input type

pub fn byte_array_custom_mapper(&mut self) -> MutVecInput<'_> {
    MutVecInput::from(&mut self.byte_array_custom_mapper)
}

pub fn byte_array_custom_mapper_corpus_extractor(&self) -> Option<&[u8]> {
    Some(&self.byte_array_custom_mapper)
}

Create mapping

let custom_mapped_mutators = havoc_mutations_no_crossover()
    .merge(havoc_crossover_with_corpus_mapper(
        &CustomInput::byte_array_custom_mapper_corpus_extractor,
    ))
    .map(ToMutVecFunctionMappingMutatorMapper::new(
        CustomInput::byte_array_custom_mapper,
    ));

Add to other mutators

// [...]
.merge(custom_mapped_mutators)
// [...]

I'm pretty sure this was not possible without the additional mapper, but this way, any input type where a part of it can be mapped to MutVecInput in a function can be used. I'd prefer to make this new mapper generic to any type II<'a>, so other owned wrapper types (such as BytesSubInput) would also work, but I'm not sure this is possible in Rust. The lifetime needs to be defined as the same as the reference to the custom input value (i.e. F: for<'a> FnMut(&'a mut IO) -> II<'a>), because otherwise the lifetime checker complains. And owned wrapper types need a lifetime for their reference to the original anyways.

Thoughts?

(Yes, I know documentation is still missing)

Sep 07 '24 20:09 riesentoaster

So the issue with MutVecInput is that it needs to be backed by a complete Vec. That's why I suggested BytesSubInput instead. It can be backed by any sort of input that exposes HasMutatorBytes and a range to point to them. If you somehow figure out how to make it work with BytesSubInput that would be ideal I think. And then we also can probably remove a bunch of other wrappers(?)

Sep 08 '24 18:09 domenukk

I finally figured out how to do this a bit more generic, for all these types that are essentially just wrappers around a reference to implement traits. This should work for both MutVecInput and BytesSubInput but also for any other wrapper type, not just for byte-array-adjacent inputs, but also AST-style or whatever.

It is built around the following new trait, which is essentially just a wrapper around its own type. There may be ways where this is not necessary, but for the life of me I could not figure out how to ensure lifetimes in the mapper without it:

pub trait WrapsReference {
    type Type<'a>
    where
        Self: 'a;
}

And an example implementation:

impl<'a> WrapsReference for MutVecInput<'a> {
    type Type<'b> = MutVecInput<'b> where Self: 'b;
}

This allows to map from such functions:

pub fn byte_array_mutvec_mapper(&mut self) -> MutVecInput<'_> {
    MutVecInput::from(&mut self.byte_array_mutvec_mapper)
}

Unfortunately, I think one of the generics has to be set manually, otherwise the compiler complains:

let custom_mapped_mutators = havoc_mutations_no_crossover()
    .merge(havoc_crossover_with_corpus_mapper(
        &CustomInput::byte_array_mutvec_mapper_corpus_extractor,
    ))
    .map(ToWrapsReferenceFunctionMappingMutatorMapper::<
        _,
        MutVecInput<'_>,
    >::new(CustomInput::byte_array_mutvec_mapper));

And then we also can probably remove a bunch of other wrappers(?)

Not quite sure what wrappers you mean. Conceptually, we may be able to write the other mappers as implementation of the two main mappers (returning &mut II and II: WrapsReference respectively), but manual implementations of FnMut require a nightly feature flag, so not sure it's possible in practice.

Sep 09 '24 13:09 riesentoaster

Ok maybe not too bad! So we can have multiple mappers for the same input, right? Could we do a blanket impl for WrapsReference for all inputs now? Maybe we should make the name more use-case specific, like MappedInput?

Sep 10 '24 07:09 domenukk

We can (and at least I will) have multiple wrappers per input. Even two ways to do so:

Add one set of mapped havoc_mutations per byte array type sub-part of the input, as demonstrated in the example fuzzer, and probably the "normal" way to do things
Add multiple sets to the same underlying structure, for advanced scheduling, e.g. to mutate the content of a structured input type more often than its header (think network packets). If modelled as a Vec<u8>, one could map the entire field to a set of havoc style mutators, and a sub-part of the vec (using a BytesSubInput) to a set as well.

I thought about a more specific name as well, but the trait itself has nothing to do with inputs, it's currently just the only place it is used. That's just my thinking, but I don't really mind it either way — whatever you prefer.

WrapsReference isn't really applicable to all input types, only those that are an owned structure around a reference. It is just there to ensure the compiler can deal with them, because their lifetime is not directly in the reference anymore, but instead a "generic" parameter (so I<'a> instead of &'a mut I).

Sep 10 '24 08:09 riesentoaster

Can I have a mapper for a WrapsReference to a new WrapsReference? Can I have a mapper that's not a WrapsReference? Are there any inputs that cannot be WrapsReference?

Sorry for the stupid questions ;) I think this might need some cleanup still, to make it super usable, but we're going in the right direction! Thx

Sep 10 '24 08:09 domenukk

Can I have a mapper for a WrapsReference to a new WrapsReference?

Yes, by passing a function that takes a reference of the origin WrapsReference. Currently there is no implementation for a mapper that takes a function of type WrapsReference -> WrapsReference. Such an implementation may be possible though.

Can I have a mapper that's not a WrapsReference?

Mappers are never WrapsReference. They may take a function that returns an implementation of WrapsReference. There is currently an implementations for mappers that take functions of type &'a mut A -> &'a mut B, and two mappers that have fixed functions mapping from &'a mut Vec<u8> to MutVecInput<'a> and from Option<T> to T. In addition of course to the new implementation that takes functions of type &'a mut A -> WrapsReference<Type=[...]<'a>.

Are there any inputs that cannot be WrapsReference?

Yes, anything that owns its data, and any reference. So for BytesInput it makes no sense to implement WrapsReference, since it does not have an externally dependent lifetime. You can still create a mapper that takes a function from &mut BytesInput and returns either a reference to something or a WrapsReference (e.g. BytesSubInput).

I think this might need some cleanup still

💯

Also: If we think this approach is flexible enough to cover any needs, I'd rather merge a more limited set of mappers now and add new implementations as the need arises, instead of creating a bunch of complicated code that never gets used.

Sep 10 '24 09:09 riesentoaster

I think we should just create havoc mutations for WrapsReference (let's call it MappedInput (or similar)?) and one for Option<MappedInput> and not provide any mappers outside of the example. If I understand correctly, there's nothing we cannot do with that setup, right?

Sep 10 '24 09:09 domenukk

Let's think about this PR in three parts:

the mapping infrastructure itself
the simplified interface for havoc_mutation mutators that uses the above (I think we may be able to get away with the two types you outlined)
the example fuzzer

And no, I don't see anything fundamentally undoable with this architecture right now. If you're happy with it, I'd do the following cleanup:

Rename WrapsReference to MappedInput
Change the simplified interface in havoc_mutations.rs to have three options:
- No mapping function, for inputs that implement HasMutatorBytes directly (so the normal havoc_mutations())
- A mapping function of type for<'a> FnMut(&'a mut IO) -> II::Type<'a> and an extractor function for the crossover mutations of type for<'b> Fn(&'b S::Input) -> Option<&'b [u8]>
- A mapping function of type for<'a> FnMut(&'a mut IO) -> Option<II::Type<'a>> and an extractor function for the crossover mutations of type for<'b> Fn(&'b S::Input) -> Option<&'b [u8]>
Remove any mappers that aren't used for the simplified interface, except the one that takes a function of type for<'a> FnMut(&'a mut IO) -> &'a mut II to retain the flexibility
Rework the example fuzzer to show the simplified interface
Add documentation where it's currently missing

Anything else?

Sep 10 '24 09:09 riesentoaster

Sounds good to me! :)

Sep 10 '24 15:09 domenukk

(Still to do: comments/unit tests for mapping mutators)

Sep 10 '24 20:09 riesentoaster

Alright, cleanup done.

The failing CI tests seem unrelated to what changed here, not sure what's up with them.

Sep 11 '24 12:09 riesentoaster

Alright:

Mapping mutators's names now include the name of their inner mutators
Users no longer have to specify types manually
mapped_havoc_mutations no longer takes a function returning an Option — this was significantly harder than I thought, because of the same issue I struggled with for the mapping mutators: lifetimes with a mix of owned objects and borrows. I ended up implementing a similar trait to MappedInput that ensures the lifetimes are correct and that includes the actual logic (so wrapping the non-Option slices with Some).

Regarding the new import path for havoc_mutations(): As discussed in person, I haven't added a pub use, since this is not deprecatable.

Thank you for your support and ideas today, btw!

Sep 18 '24 20:09 riesentoaster

You can fix the names in a follow-up PR. Great work :)

Sep 18 '24 21:09 domenukk