stedolan
avoid `Premature end of file`
Perhaps I'm not using correctly crowbar, but it seems that creating a correct test case is not user friendly: an empty file is not enough and I don't know if depending on the test cases you have to create a file arbitrary big. Instead of premature end of file couldn't crowbar consider that it is filled with null bytes?
PS: the makefile in examples should have examples that uses afl-fuzz even if they are not run by default. Except that crowbar is great!
@bobot, did you run into this problem when running a Crowbar test under afl-fuzz? I had a problem where afl-fuzz would report one unique crash after about 3,500 runs, but when I reran the test program with the crash input, I would get premature end of file and the program would exit normally. The actual crash was caused by something else, unrelated to the file length, which crowbar and afl-fuzz were handling correctly, but which didn't turn up in a single run.
I was able to fix it in my environment by changing a constant in afl-persistent. Could you try pinning afl-persistent with this change (opam pin add afl-persistent https://github.com/yomimono/ocaml-afl-persistent.git#persist-less) and see whether you get more sensible behavior?
We recently ran into this with @pascutto. This works well with afl-fuzz which doesn't consider that as a crash but it can be slightly disturbing when debugging other issues.
I think a slightly improved error message in this case advising the user to provide larger input would help!
I'm happy to provide a patch for that!
As a side note, I wonder how AFL behaves in those cases and if reading 0x00 bytes once we reached the end of the input —as @bobot suggests— would make the fuzzing more efficient.
A better error message is a good idea.
Reading 0x00 bytes won't help, though. Sometimes an arbitrarily long sequence of null bytes is a valid prefix of a test case, so silently supplying nulls will turn a failure into nontermination.
Could we generate a random sequence of bytes in this case, the probability of non termination would be smaller.
No, often the probability of non-termination is high even for a random sequence of bytes. (See section 3.2 of Claessen & Hughes' original QuickCheck paper)