S3Boto3Storage - "PermissionError: Access Denied" persisted when accessing a file before it is created.

[seanchon]

Hello,

First off, I just want to say thank you. This package has been extremely helpful for a project that I am working on.

I am using this package to store pandas dataframes to S3, and I have come across a case where a PermissionError: Access Denied is persisted if I:

1. attempt to access a file that does not exist import pandas as pd; pd.read_csv("s3://media_root_dir/test.csv")
2. later create the file (the process I am using to create a dataframe can be a large computation, so this can be delayed)
3. attempt to access the file again in the same session (step 1)

If I restart my session (Django shell or webserver), I am able to access the dataframe. The implication is that I am serving some dataframes via Django REST Framework, and if I attempt to access the dataframe before it is created, I cannot later access the dataframe after it has actually been created.

I have subclassed S3BotoStorage as follows:

from django.conf import settings

from storages.backends.s3boto3 import S3Boto3Storage


class MediaStorage(S3Boto3Storage):
    """
    Custom storage for collecting user-generated media files to dedicated
    S3 bucket.
    """

    bucket_name = settings.AWS_MEDIA_BUCKET_NAME
    location = settings.MEDIA_ROOT_DIR

Thank you for any assistance!

Sean