website icon indicating copy to clipboard operation
website copied to clipboard

Optimize replay file deletion

Open tsa96 opened this issue 2 years ago • 0 comments

We'll be deleting a lot of replay files from S3, when a user PBs, and when maps are updated (changing state during map review, patching a map to fix a catastrophic bug). In the latter case we'd be reaaaaally hammering S3/R2 if we do individual deletes.

Also, we're going to be adding a status flag to each run in the future, likely PENDING_VALIDATION, APPROVED. Can easily add another, PENDING_DELETION. When a run is deleted, we just update the service to PENDING_DELETION.

Then, have a new service that uses NestJS's @Cron decorator from the scheduler that periodically, within an interactive transaction gets all runs marked as PENDING_DELETION, creates a S3 delete multiple call, and assuming that's successful, actually deletes all those runs from DB, interactive transaction ends.

tsa96 avatar Jul 05 '23 23:07 tsa96