Caching with pre-signed url

[pljspahn]

The pre-signed url signature changes each time a file is requested since it's always created at a different moment.

Can I cache the pre-signed url so that it's not generated for each request? I assume in a manner like is described is this SO discussion - https://stackoverflow.com/questions/19583338/s3-signed-urls-and-caching

I haven't looked too deep into boto3, so this is maybe something that would be too awkward if not already implemented there.

Can django-s3-storage possibly just use built-in Django cache to cache the pre-signed url passing along the expiration as set with AWS_S3_MAX_AGE_SECONDS?

[etianen]

Does this work for you?

https://github.com/etianen/django-s3-storage#optimizing-media-file-caching

On Fri, 18 Mar 2022 at 20:40, pljspahn @.***> wrote:

The pre-signed url signature changes each time a file is requested since it's always created at a different moment.

Can I cache the pre-signed url so that it's not generated for each request? I assume in a manner like is described is this SO discussion - https://stackoverflow.com/questions/19583338/s3-signed-urls-and-caching

I haven't looked too deep into boto3, so this is maybe something that would be too awkward if not already implemented there.

Can django-s3-storage possibly just use built-in Django cache to cache the pre-signed url passing along the expiration as set with AWS_S3_MAX_AGE_SECONDS?

— Reply to this email directly, view it on GitHub https://github.com/etianen/django-s3-storage/issues/138, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABEKCDB7KYLTTKSWQO4LITVATSKLANCNFSM5RC3GHLQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

[pljspahn]

Doesn't that merely disable private bucket access?

I was looking for a solution that keeps private buckets and instead of generating the presigned url on every request, the url is either:

1. Generated once, cached with default Django cache features, and kept until it expires, or
2. Generated from a fixed moment in time (as described in the SO link I posted above, ie "tomorrow at 12 am")

[etianen]

You'd need to subclass the storage backend to add that feature.

Beware though - caching the URL for an extended period effectively makes it public. The default token expiry is an hour.

On Mon, 21 Mar 2022 at 17:10, pljspahn @.***> wrote:

Doesn't that merely disable private bucket access?

I was looking for a solution that keeps private buckets and instead of generating the presigned url on every request, the url is either:

1. Generated once, cached with default Django cache features, and kept until it expires, or
2. Generated from a fixed moment in time (as described in the SO link I posted above, ie "tomorrow at 12 am")

— Reply to this email directly, view it on GitHub https://github.com/etianen/django-s3-storage/issues/138#issuecomment-1074167148, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABEKCDU4EAL54NT7JQIY7TVBCT4JANCNFSM5RC3GHLQ . You are receiving this because you commented.Message ID: @.***>

[pljspahn]

I'll look through the code a little more, but I'm not really sure how the token expiration is even used. The url returned from https://github.com/etianen/django-s3-storage/blob/38f6fbfd143088b7615f63bd084d5ef02b1e47c6/django_s3_storage/storage.py#L428

seems to be generated on every request. So even if the signed url is valid for one hour, the next time the page is loaded, the url returned is a new signed url, which would effectively ignore the expiration since the previous signed url isn't cached anywhere and disappears into the ether.

[etianen]

You'd want to wrap and cache the 'url' method, and increase the token expiry time.

On Fri, 25 Mar 2022 at 20:50, pljspahn @.***> wrote:

I'll look through the code a little more, but I'm not really sure how the token expiration is even used. The url returned from https://github.com/etianen/django-s3-storage/blob/38f6fbfd143088b7615f63bd084d5ef02b1e47c6/django_s3_storage/storage.py#L428

seems to be generated on every request. So even if the signed url is valid for one hour, the next time the page is loaded, the url returned is a new signed url, which would effectively ignore the expiration since the previous signed url isn't cached anywhere and disappears into the ether.

— Reply to this email directly, view it on GitHub https://github.com/etianen/django-s3-storage/issues/138#issuecomment-1079421177, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABEKCAPXANOGH23S3QY6SDVBYQVZANCNFSM5RC3GHLQ . You are receiving this because you commented.Message ID: @.***>

[pljspahn]

I ended up adding a url property to my image model:

from django.core.cache import cache

class MyImage(models.Model):
    parent = models.ForeignKey(ParentModel, related_name='images', on_delete=models.CASCADE)
    image = models.ImageField(upload_to=assign_sign_sku)

    @property
    def url(self):
        return cache.get_or_set(self, self.image.url, 1440)

[etianen]

Perfect!

On Thu, 7 Apr 2022 at 19:58, pljspahn @.***> wrote:

I ended up adding a url property to my image model:

from django.core.cache import cache

class MyImage(models.Model): parent = models.ForeignKey(ParentModel, related_name='images', on_delete=models.CASCADE) image = models.ImageField(upload_to=assign_sign_sku)

@property
def url(self):
    return cache.get_or_set(self, self.image.url, 1440)

— Reply to this email directly, view it on GitHub https://github.com/etianen/django-s3-storage/issues/138#issuecomment-1092091024, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABEKCHB6WOBADP5JDNK6ADVD4VIVANCNFSM5RC3GHLQ . You are receiving this because you commented.Message ID: @.***>