LibAFL icon indicating copy to clipboard operation
LibAFL copied to clipboard

WIP: Qemu full system mode

Open bitwave opened this issue 3 years ago • 6 comments

A working (but unorthodox, as it uses klo routines) QEMU full system fuzzer using LibAFL+kloroutines. It currently supports only aarch64 and a simple interface via svc #0xaf1 to use multiple hypercalls for communication. Notable to mention is, that libafl is "pulled" from the fuzzer part inside the QEMU guest instead of controlling QEMU directly. This makes it possible to integrate libafl into a normal QEMU rather simple, as QEMU uses an multi-threaded, async architecture for full-system emulation.

The code is far from perfect an probably needs some cleanups & improvements.

bitwave avatar Feb 22 '22 17:02 bitwave

If it's not possible with a normal executor like in the usermode qemu case, we should still be able to move this to PushMutational stage instead of kloroutines, see https://github.com/AFLplusplus/LibAFL/blob/ef01009f30af40ccda01ed1b0ea15de550bdcf64/fuzzers/push_stage_harness/src/main.rs#L109

domenukk avatar Feb 22 '22 17:02 domenukk

A working (but unorthodox, as it uses klo routines) QEMU full system fuzzer using LibAFL+kloroutines. It currently supports only aarch64 and a simple interface via svc #0xaf1 to use multiple hypercalls for communication. Notable to mention is, that libafl is "pulled" from the fuzzer part inside the QEMU guest instead of controlling QEMU directly. This makes it possible to integrate libafl into a normal QEMU rather simple, as QEMU uses an multi-threaded, async architecture for full-system emulation.

The code is far from perfect an probably needs some cleanups & improvements.

How difficult is to merge it into libafl_qemu in your opinion? Would be cool to support both breakpoints and hypercalls

andreafioraldi avatar Feb 25 '22 09:02 andreafioraldi

The biggest point against merging is, fullsystem qemu is highly async/multithreaded in comparision to single threaded usermode qemu. To seperate these differences I chose to use a new crate (reusing parts of libafl_qemu).

In my concrete use-case I need to boot a Linux system, which then calls another OS in ARM secure mode. And inside this other OS I start fuzzing. Therefore I need to know where to set a breakpoint a priori, which is not trivial, and chose the simple way using a hypercall, which triggers the fuzzer. My main fuzz loop logic is setup inside the guest :D If i detect a crash, I simply load a snapshot and the guest fuzz logic starts again. Maybe not the best design, but the minimal invasive one, which is implementable inside of fullsystem qemu.

bitwave avatar Feb 26 '22 10:02 bitwave

The biggest point against merging is, fullsystem qemu is highly async/multithreaded in comparision to single threaded usermode qemu. To seperate these differences I chose to use a new crate (reusing parts of libafl_qemu).

In my concrete use-case I need to boot a Linux system, which then calls another OS in ARM secure mode. And inside this other OS I start fuzzing. Therefore I need to know where to set a breakpoint a priori, which is not trivial, and chose the simple way using a hypercall, which triggers the fuzzer. My main fuzz loop logic is setup inside the guest :D If i detect a crash, I simply load a snapshot and the guest fuzz logic starts again. Maybe not the best design, but the minimal invasive one, which is implementable inside of fullsystem qemu.

After some months, I started to look into fullsystem for libafl today. I agree that the hypercall must be way to go, but why it can't simply behave like a breakpoint (they stop the CPU loop and return the control to the caller)? Also, fuzzing and multithreading are not friends, can't we just use one thread in qemu but emulate several CPUs? IIRC the smp flag allows you to specify more emulated CPUs than the cores available on the host, so it's not one emulated core == one host core.

andreafioraldi avatar Jun 16 '22 10:06 andreafioraldi

Is this superseded by #692 ?

domenukk avatar Jul 29 '22 09:07 domenukk

Is this superseded by #692 ?

yes but I will close this after having a full system fuzzer

andreafioraldi avatar Jul 29 '22 09:07 andreafioraldi

I think this is now superseded by #883, and can be closed, correct?

domenukk avatar Dec 04 '22 18:12 domenukk