LibAFL icon indicating copy to clipboard operation
LibAFL copied to clipboard

Object-achieving results should be accompanied by machine- and human-readable metadata

Open s1341 opened this issue 4 years ago • 3 comments

When objective-achieving results are saved, they should include all the metadata, mutator and observer data that is available related to this test-case.

Perhaps this can be achieved by storing a json/toml/yaml file next to the test-case input, with the same base name.

At a minimum the following information should be present:

  • the test-case's unique name
  • the date/time at which it was encountered/produced
  • the type of objective-achievement
  • the arguments/cmd-line which was used to generate the test-case
  • a cmd-line which can be used to reproduce the test-case
  • mutations that were applied to generate the test-case
  • the parent test-case which was mutated
  • observer output

s1341 avatar Apr 11 '21 11:04 s1341

@andreafioraldi has started work on this, by adding the ability to append metadata from Observers to solutions.

s1341 avatar Apr 23 '21 11:04 s1341

What parts are missing here?

domenukk avatar Apr 28 '21 06:04 domenukk

We still don't have unique names for the test-cases. We also don't have 'default metadata' including timestamp, cmd-line used to generate the test case, mutations applied, parent case etc.

s1341 avatar Apr 28 '21 06:04 s1341

We have a good amount of metadata now. We could add more like mutations applied, parent case, cmd-line, but for now I'm closing this issue

domenukk avatar Jan 26 '23 01:01 domenukk