hedy
hedy copied to clipboard
hedyorg
[PERFORMANCE] Change users table structure to enable pagination and filtering
This PR is not for merging, only for review and than running once to make sure everything works.
Since this PR is open for quite some time already, I will shamelessly leave my unsolicited review here.
The script does successfully add an 'epoch' property to every user. The requests are not paced, which means that we might get throttled by AWS if we have many users to update. There is no error handling, but the update is idempotent, so even if the script fails halfway through, you could just rerun it (at the expense of extra read and write operations). I did execute it against my AWS dynamo setup (which mirrors Hedy's) and it was successful.
Unfortunately, I don't have the context of why we are adding an 'epoch' property but perhaps this is something @rix0rrr can comment on.
Hi @boryanagoncharenko! Thanks for checking this PR, I had already forgotten it existed. The goal of adding an epoch is to enable us pagination on the sort key timestamp without needing to filter on an additional index key. Sadly Dynamo doesn't support the statement "give me the last x programs, sorted by timestamp".
In this case we need to provide an index key as well, by giving all programs a dummy value op epoch: 1 we can call all programs with the index epoch: 1 (which is all programs in this case) and then use the timestamp as sort key. Enabling us to efficiently sort all programs for the /explore page.
Looking into this we might also be able to use PartiQL, which is supported within the DynamoDB web interface. Following the documentation we might be able to use some UPDATE statement: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ql-reference.update.html.
Maybe something like this?
UPDATE "hedy-alpha-users"
SET epoch=1
Looking into this we might also be able to use PartiQL, which is supported within the DynamoDB web interface. Following the documentation we might be able to use some UPDATE statement: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ql-reference.update.html.
Maybe something like this?
UPDATE "hedy-alpha-users" SET epoch=1
Nevermind....
@Felienne I think we can run this on the Alpha db if that's okay with you. Than after some extensive testing (and changing a few db calls on the admin side) we can run it again on the Beta db.
