rematch
rematch copied to clipboard
Mechanism to test accuracy and precision of engines
In order to properly develop decent engines, it would be convenient to have a easy way to benchmark and test results of available engines against a properly-sized labeled dataset.
- [ ] Collect a properly-sized labeled dataset.
- [ ] Create a mechanism to easily test engines against said set (without going through the IDA UI manually, see #198 ).
- [ ] Include as part of tests to streamline Engine creation (kinda requires IDA running in CI).
Those are two out of three binaries from the old manual test set we used. Third is >50M compressed and above the 10M limit github has.