open-humans icon indicating copy to clipboard operation
open-humans copied to clipboard

Store filesize in database

Open madprime opened this issue 5 years ago • 0 comments

Whenever the property size is called on files, we're currently doing a call to S3 to retrieve the file's actual size -- which can slow things down a lot, e.g. when trying to list a bunch of files along with their sizes.

Suggested improvement: add a field size to store this value (a permanent cache of the value from S3). A call to S3 can be done when there's good cause to double-check the value.

Note: be wary of any potential places in code where the size call was been used as a proxy for checking the presence of a file on S3!

madprime avatar May 28 '19 18:05 madprime