sv-benchmarks icon indicating copy to clipboard operation
sv-benchmarks copied to clipboard

Ping original submitters to fix errors spotted by continuous integration checks

Open tautschnig opened this issue 10 years ago • 5 comments

Hi, In #4, #5, #6 I have provided patches for those benchmarks that I have submitted or feel somewhat responsible for. check-blacklist nevertheless still lists 3560 broken benchmarks, which ought to be fixed before the next edition of the competition commences. It would be great if the original submitters of the benchmarks could be pinged to review and fix their submissions.

To help people understand what is going wrong with their specific files, they should run

cd c ; ./check <directory-of-interest>/*.[ci]

Thanks a lot, Michael

tautschnig avatar Sep 11 '15 08:09 tautschnig

I think this can be closed.

PhilippWendler avatar Nov 01 '15 18:11 PhilippWendler

See also my rambling on the SV-COMP mailing list just moments ago: unless #56 and #59, plus #106 are closed, it would likely remain a good idea to actually ping people. For any benchmarks submitted since the move to github this would be task that anyone could take care of; before, however, most benchmarks were committed by @dbeyer, who clearly is not to be blamed for the syntax errors in them...

Best, Michael

tautschnig avatar Nov 07 '15 23:11 tautschnig

While the mentioned issues have been fixed, there are still several open problems, where we might need to contact the original submitters.

Lists of open issues are

PhilippWendler avatar Nov 14 '16 10:11 PhilippWendler

I've tried to do at least the administrative changes in #239. I believe that all the remaining ones of check.py are already being tracked as issues, except for termination benchmarks marked "unknown-termination".

tautschnig avatar Nov 26 '16 21:11 tautschnig

The single PR #239 has now been replaced by individual ones in #243..#260.

tautschnig avatar Nov 27 '16 17:11 tautschnig