Judge error in external diff

[meisterT]

trafficstars

Kattis exposed a judge error to us, which was not shown in the external diff (probably because the submission was still judging on our side). We might want to extend the filter to show also judging submissions on our side to see how far we are behind Kattis.

[meisterT]

@nickygerritsen I think this can be closed?

[nickygerritsen]

Can it though? I think if they expose a JE we don't import it, since we can't map it to one of our verdicts from verdicts.php.

[meisterT]

I assumed they would be merged into the still judging (aka unknown result) runs, but perhaps I am mistaken.

[nickygerritsen]

I believe not but we could do that

[vmcj]

We should think about mapping all possible verdicts to our verdicts as in the case where you shadow you should be more open to all choices.

@nickygerritsen suggested to extend verdicts.php to an external version (only for shadowing) so this can show up in the matrix with an indicator that we mapped to a different verdict according to a possible CCS spec.

[nickygerritsen]

Yes and then add some notice but not a 'real' shadow difference. For example we map TNO to TLE

[nickygerritsen]

If we do #1849, this will also be possible, so let's close this one.