rematch icon indicating copy to clipboard operation
rematch copied to clipboard

Mechanism to test accuracy and precision of engines

Open nirizr opened this issue 8 years ago • 1 comments

In order to properly develop decent engines, it would be convenient to have a easy way to benchmark and test results of available engines against a properly-sized labeled dataset.

  • [ ] Collect a properly-sized labeled dataset.
  • [ ] Create a mechanism to easily test engines against said set (without going through the IDA UI manually, see #198 ).
  • [ ] Include as part of tests to streamline Engine creation (kinda requires IDA running in CI).

nirizr avatar Feb 05 '17 10:02 nirizr

Those are two out of three binaries from the old manual test set we used. Third is >50M compressed and above the 10M limit github has.

zues_variants.zip file_dump.zip

nirizr avatar Feb 21 '17 13:02 nirizr