Balloon list may have problem on awarding the "first to solve"

[bhscer]

Description of the problem

I'm intersted in the logic of balloon.

I found that when the submission judged as Accepted, the balloon will be created immediately. https://github.com/DOMjudge/domjudge/blob/7c890e8ddf0ed13c6164e091d903408736f465a8/webapp/src/Controller/API/JudgehostController.php#L1078

But if we have two submission on the same problem (they are the eariliest submissions of the problem) , maybe we call them A and B, and the submit time of A is earlier than B, and A's judge finish time is later, the balloon of submission A will be created after submission B. But when we looking at the balloon list, we temporally found that the balloon of submission B is remindered as the first balloon. https://github.com/DOMjudge/domjudge/blob/7c890e8ddf0ed13c6164e091d903408736f465a8/webapp/src/Service/BalloonService.php#L92
In the collectBalloonTable function, we can see that award is determined by the balloon list

The above is my understanding of the balloon logic.

I don't know whether my worry is right or not. If it is, I think it is a bug or if you can tell me how can I temporaly solve this problem (maybe we should send balloons only if there submit time is much earlier than now to avaid eariler submission is not judged yet ?)

Your environment

Steps to reproduce

Create a problem with a long time limit. Team A submit a code correct but run very slow first. Team B submit a code correct and run fast later.

Expected behaviour

It's obvious Team A is the first to solve.

Actual behaviour

When Team B's submission is judged as correct but Team A's submission is still judging, we can found the balloon list maked Team B is the first to solve the problem.

Any other information that you want to share?

Sorry that currently I have no machine to test it. But according to my understing, I think this bug truely happens.

[meisterT]

I can indeed reproduce this by disabling judging for a language, submitting a correct solution as a team in that language, and then with another team in a language that is enabled for judging. This will be shown as first in contest.

[meisterT]

We had some more discussion about this. The attached PR doesn't fix the problems for first in contest and it is kinda hard to fix (also should be generalized in award service). Also for first in problem, after the PR, we would not display "first in problem" until we are sure that it actually is the "first in problem", but display the balloon without it. So someone could deliver the balloon without realizing it actually could be or is the first in problem. One option would be to hold back the balloon until the info is known (which could be a long time assuming that a compiler/language could be broken) and confusing even when it's not a long time.

So no perfect solution. The question is whether we really should fix it or just remove the feature as it is hard to get it right.

I went and did some code archaeology, and found that the feature was added as part of a CCS requirement which no longer exists, see https://github.com/DOMjudge/domjudge/issues/70

I am leaning towards removing first in contest and first in problem on the balloon page. If people would want that functionality, they could use the balloon printer in ICPC tools.

[bhscer]

We had some more discussion about this. The attached PR doesn't fix the problems for first in contest and it is kinda hard to fix (also should be generalized in award service). Also for first in problem, after the PR, we would not display "first in problem" until we are sure that it actually is the "first in problem", but display the balloon without it. So someone could deliver the balloon without realizing it actually could be or is the first in problem. One option would be to hold back the balloon until the info is known (which could be a long time assuming that a compiler/language could be broken) and confusing even when it's not a long time.

So no perfect solution. The question is whether we really should fix it or just remove the feature as it is hard to get it right.

I went and did some code archaeology, and found that the feature was added as part of a CCS requirement which no longer exists, see #70

I am leaning towards removing first in contest and first in problem on the balloon page. If people would want that functionality, they could use the balloon printer in ICPC tools.

But I should appologize first that I misunderstanding the logic of "first for problem". In my newest understanding, the 'first to contest' have some problem, and need fix or remove, but the 'first to solve' comes from the scoreboard, why we should remove it?

[meisterT]

See this snippet:

Also for first in problem, after the PR, we would not display "first in problem" until we are sure that it actually is the "first in problem", but display the balloon without it. So someone could deliver the balloon without realizing it actually could be or is the first in problem. One option would be to hold back the balloon until the info is known (which could be a long time assuming that a compiler/language could be broken) and confusing even when it's not a long time.

Basically, the problem is either we make the info mutable (as in add "first in problem" once we are sure it is first) or hold the balloon back which is confusing and also not great (and would require to hold back all other balloons for the same problem even the ones which cannot be first to solve in problem).

[bhscer]

So you means that it maybe a case that a balloon is 'first for problem' but we don't sure so we don't show this award on the balloon list?

But we can find that there are two cases:

1. This submission is the first to submit for the problem and judge finished firsted eariest.
2. This submission is the first to submit for the problem but judge finished later than some later submissions.

But whatever the case is, for this truely 'first for problem' submission, we update scoreboad before insert the balloon, so in balloon list we can immedately see 'first for problem'.

[bhscer]

Ok, I'm sorry that I forgot another case. If early incorrect submission judge finished later than this truely first for problem submissuon, then the scoreboard will decide first for problem later than the balloon insert. So that means current we have to problem:

1. 'first for problem' may not display immedately on the list
2. 'first to contest' may be incorrect (fixed)

Maybe we should change the logic of balloon insert. After a judge X for problem A finished as correct, we should query if there are some submission earlyer than this one but not finish the judge. If such submission exist, we skip the insert process, otherwise, we can process those submissions (for problem A, submit time <= X.submit time) to insert many balloons.

[nickygerritsen]

We discussed this again and we decided we don't think as maintainers we want to implement this correctly, since it is clearly non trivial. We will remove the feature and if people are interested in it, they are welcome to implement it themselves and we can re-evaluate if we want to integrate this.