matrix-media-repo icon indicating copy to clipboard operation
matrix-media-repo copied to clipboard
verified publisher icon t2bot

S3 prefixes

Open Fizzadar opened this issue 3 years ago • 1 comments

We are sometimes seeing S3 rejecting upload requests from the media repo, some investigation leads to https://stackoverflow.com/questions/52443839/s3-what-exactly-is-a-prefix-and-what-ratelimits-apply which implies prefixes would help (of course being AWS the documentation is... lacking!).

Currently object keys are just random hashes and thus all share the same non-prefix. Should be safe to make the change to split that into a prefix/key and it'll work for any future media uploads.

I've got a WIP branch adding this as a datastore option, adding this issue for that.

Fizzadar avatar Feb 18 '22 11:02 Fizzadar

@LisoUseInAIKyrios, anything holding back from this getting a PR?

everpcpc avatar Feb 14 '23 01:02 everpcpc