Adding new milestone breaks the site

[ksy36]

I've tried to deploy a new milestone on staging today and after deployment, the site was showing 500 error.

Worth mentioning that I've created "moved" milestone on GitHub last week.

After inspecting the service logs it looks like there is a mismatch in milestones:

ksenia@staging-server-webcompat:/home/webcompat/vassals$ sudo systemctl status webc-staging
● webc-staging.service - uWISGI staging.webcompat.com
     Loaded: loaded (/etc/systemd/system/webc-staging.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2021-11-22 19:22:54 UTC; 24min ago
    Process: 431969 ExecStart=/usr/local/bin/uwsgi --ini /home/webcompat/vassals/staging-uwsgi-conf.ini --touch-reload=/tmp/uwsgi-staging-reload.ini --plugin python38 (code=exited, status=1/FAILURE)
   Main PID: 431969 (code=exited, status=1/FAILURE)

Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]: added /home/webcompat/staging.webcompat.com/env/lib/python3.8/site-packages/ to pythonpath.
Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]: Writing logs to: /home/webcompat/staging.webcompat.com/tmp
Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]: Statuses Initialization…
Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]: A milestone is missing or has been added: ['moved']
Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]:
                                                                          Check the milestones names on your Github repository and try again.
                                                                          This error was probably caused by a typo.
                                                                          Your milestones.json was erased and a backup copy was created at /home/webcompat/staging.webcompat.com/data/milestones_backup.json.
Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]:
Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]: Milestones are not initialized. Check logs.
Nov 22 19:22:54 staging-server-webcompat webcompat-staging-error[431969]: VACUUM WARNING: unix socket /tmp/uwsgi-staging.sock changed inode. Skip removal
Nov 22 19:22:54 staging-server-webcompat systemd[1]: webc-staging.service: Main process exited, code=exited, status=1/FAILURE
Nov 22 19:22:54 staging-server-webcompat systemd[1]: webc-staging.service: Failed with result 'exit-code'.

So it looks like if data/milestones.json exists, we won't fetch the updates from GitHub: https://github.com/webcompat/webcompat.com/blob/67f2831bd6a12a157a44ec165590f0b8edb03c31/config/init.py#L53-L79

However, there is a check in this update_status_config function whether STATUSES keys match the data/milestones.json. Since the new moved milestone is not in the json file, the app fails to start:

https://github.com/webcompat/webcompat.com/blob/67f2831bd6a12a157a44ec165590f0b8edb03c31/config/init.py#L98-L119

But data/milestones.json wouldn't get the new milestone without fetching it from GitHub first, so the fact that it's not matching makes sense. Did I need to delete the file prior to deploying, so the new milestones will be fetched from scratch?

Wonder if I missed something, or is this a bug. @karlcow do you have an idea?

[karlcow]

yes this code has never been great (my bad). It was meant to isolate and make reusable elsewhere without hardcoding. If I had to redo it today I would probably create the json milestone file at the deployment phase.

In the deploy script:

1. fetch the milestones
2. if fetch fails, bail out with an error message
3. save the milestones as json in data/
4. continue to deploy the app.

in the app:

1. check if data/milestones.json exist

PS: if I was rewriting it today I would also redo the Config altogether with a model closer to what is in the documentation now.

[ksy36]

Thanks, @karlcow, I'll look into this.