Archipelago icon indicating copy to clipboard operation
Archipelago copied to clipboard

WebHost: add startup option to clean datapackages

Open Berserker66 opened this issue 1 year ago • 6 comments

What is this fixing or adding?

Allows cleaning datapackage entries that have the wrong checksum. They're deleted entirely, which should be fine for the server, though some clients may break. This is a very small may though, as it should only affect clients that relied on a miscontructed apworld.

How was this tested?

copied the checksum from one row and the data from another row, then watched how only that row disappeared on startup.

If this makes graphical changes, please attach screenshots.

Berserker66 avatar Apr 24 '24 03:04 Berserker66

Do you think this should also be added to docs/webhost configuration sample.yaml?

ThePhar avatar Apr 24 '24 03:04 ThePhar

No idea, if you think it should, feel free to PR it in.

Berserker66 avatar Apr 24 '24 04:04 Berserker66

Additional clarification was made in Discord, so going to post it here so it's not lost.

Essentially found out this check will not work for the issue described in #3206 because upload.py ends up overriding the checksum on both the column and in the data package of the uploaded file with the re-calculated value so it's essentially impossible to look for if the rest of the data package is technically valid.

Leaving my approval, but unless someone can think of another reason this could be useful (maybe there was another way data packages could be corrupt?), I don't think this will be necessary.

ThePhar avatar Apr 24 '24 04:04 ThePhar

I feel like this should be a click instead, so you can just run flask cleanup-database in a separate process in the same venv without impacting the running instance.

black-sliver avatar Apr 24 '24 06:04 black-sliver

As for what to clean out, I think it should (also) search for unreferenced game packages, so that when seeds are deleted we also delete the corresponding games packages on the next cleanup. (Since this needs to load all rooms and create a massive set, I don't think that can be done on startup, but needs to run in a separate process.)

black-sliver avatar Apr 24 '24 06:04 black-sliver

As for what to clean out, I think it should (also) search for unreferenced game packages, so that when seeds are deleted we also delete the corresponding games packages on the next cleanup. (Since this needs to load all rooms and create a massive set, I don't think that can be done on startup, but needs to run in a separate process.)

Yeah, I think that should be doable.

ThePhar avatar Apr 24 '24 13:04 ThePhar