WinAFL fails to kill child when using WinIPT mode

[vmcall]

I have been dealing with an issue where winafl could not fuzz binaries in Intel PT mode if they took input files by command-line and did not close handles to the input file. To reproduce this error, I used the following binary:

#define _CRT_SECURE_NO_WARNINGS
#include <iostream>
int main(int argc, char** argv)
{
    unsigned char data;
    FILE* f = fopen(argv[1], "rb");
    fread(&data, 1, 1, f);

    // Uncommenting this will make it work
    //fclose(f);
}

And fuzzed using: afl-fuzz.exe -i in -o out -P -t 20000 -- -coverage_module test.exe -target_module test.exe -target_offset 0x1000 -nargs 2 -- test.exe @@

This will instantly fail in afl-fuzz.c!write_to_testcase during the dry run as it can't create "out/.cur_input." because the child process still has an open handle to the input file (WinAFL uses O_EXCL which will fail in that case)

Debugging afl-fuzz.c!write_to_testcase shows that the afl-fuzz.c!destroy_target_process call instantly returns as the global child_handle is null. This is supposed to be set in afl-fuzz.c!create_target_process which should be called by afl-fuzz.c!run_target, but it is not as using Intel PT will make afl-fuzz.c!run_target simply call run_target_pt as seen here.

This can be further proved by making afl-fuzz.c!write_to_testcase repeatedly call afl-fuzz.c!destroy_target_process until the child has exited. This causes an infinite loop as afl-fuzz.c!destroy_target_process is essentially a nop in WinIPT mode.

A dirty workaround for people having this issue is to change the open mode to be non-exclusive:

if (out_file) {
  unlink(out_file); /* Ignore errors. */
  fd = open(out_file, O_WRONLY | O_BINARY | O_CREAT, 0600); // Non exclusive
  if (fd < 0) {
    destroy_target_process(0);
    
  unlink(out_file); /* Ignore errors. */
  fd = open(out_file, O_WRONLY | O_BINARY | O_CREAT, 0600); // Non exclusive
	
    if (fd < 0) PFATAL("Unable to create '%s'", out_file);
}

This will make winafl not care about potential third-party handles to the input file.

[ifratric]

You are correct that destroy_target_process is not appropriate here, in the Intel PT mode the correct function to call would be https://github.com/googleprojectzero/winafl/blob/master/winaflpt.c#L1381 (or, better yet, destroy_target_process should be modified to call that in the PT case)

Furthermore, not closing the input handle would be an issue not just for PT mode but for other modes as well. It seems O_EXCL was carried over from upstream AFL, not sure if there's a good reason for it to be there.

Since you seem to be in a good position to test these, would you like to submit a pull request? :-)

[ifratric]

I should note, however, that not closing the input handle in the target might lead to resource problems on the machine, as you would be leaking one file handle per persistent iteration.