django-storages icon indicating copy to clipboard operation
django-storages copied to clipboard

Cache S3 key metadata to avoid unnecessary calls especially with collectstatic

Open kupsum opened this issue 2 years ago • 3 comments

Relates to https://github.com/jschneier/django-storages/issues/1255

When running collectstatic S3 key check first the existence and the last modification time. With this change the latter call is avoided. Previously last modification fetched the whole object and didn't use the metadata. This cause a lot of unnecessary traffic and thus making collectstatic slow.

kupsum avatar Mar 16 '24 09:03 kupsum

Is there any chance we can move this issue forward? :)

davidmir avatar Apr 15 '24 16:04 davidmir

Does this work in a multi threaded environment or does the cache need a lock?

jschneier avatar Jun 09 '24 22:06 jschneier

+1 for this PR, will really increase a speed of my deployment :)

@jschneier, from their docs cachetools are not thread-safe (https://cachetools.readthedocs.io/en/stable/)

LeaveMyYard avatar Aug 05 '24 20:08 LeaveMyYard