codalab-competitions icon indicating copy to clipboard operation
codalab-competitions copied to clipboard

"Failed, competition phase is being migrated, please try again in a few minutes"

Open Didayolo opened this issue 2 years ago • 3 comments

This error message has been reported multiple times:

Failed, competition phase is being migrated, please try again in a few minutes
  • #3190
  • #3422
  • #3418
  • #3064
  • #3410

Cross-domain MetaDL

https://codalab.lisn.upsaclay.fr/competitions/5635

For the second phase, the last submission of the first phase should be automatically forwarded. Therefore, in the second phase I selected the auto migration option. However, sometimes no submissions can be done because the platform displays the error message.

L2RPN Delft 2023

https://codalab.lisn.upsaclay.fr/competitions/12420

The same error happened in this competition, after the incident (crash and restart of the platform) of this week-end (#3424). Several users reported that they could not submit to this competition anymore. No all competition are affected. The settings of auto-migration should be investigated.

2023 IEEE VIP Cup

https://codalab.lisn.upsaclay.fr/competitions/14006

This problems has been reoccurring for the past couple of days. I have removed the "auto-migration" options between phases as well as deleted all "failed" and "running" submissions. However this problem still persists.

Didayolo avatar Aug 28 '23 14:08 Didayolo

The bug appears soon after clicking to Submit. The submission is not even read.

Capture d’écran 2023-08-28 à 16 12 11

Didayolo avatar Aug 28 '23 14:08 Didayolo

Hi, thanks for checking the bug. Do you maybe have an estimate that when the bug will be fixed? Since we are close to the deadline of our competition (L2RPN Delft 2023), and participants are asking me about it. thanks

AliRajaei95 avatar Aug 29 '23 12:08 AliRajaei95

@AliRajaei95 The three competitions mentionned in this issue should now be working fine now (L2RPN, MetaDL, VIP Cup).

The problem is that the is_migrating_delayed competition property sometimes is set to True but never set it back False.

This needs to be fixed in the code. The problem is when the phase is migrated while a submission is running. Let's keep this issue open while this problem is not fixed.

In the mean time, we can fix manually the competitions by going inside the production Django container, then using the shell_plus:

>>> comp = Competition.objects.get(id=<ID>)
>>> comp.is_migrating
False
>>> comp.is_migrating_delayed
True
>>> comp.is_migrating_delayed = False
>>> comp.save()

Didayolo avatar Aug 31 '23 14:08 Didayolo