s3fs icon indicating copy to clipboard operation
s3fs copied to clipboard

Reading errors after upgrade from 0.2.0 to 0.4.0

Open ohadmata opened this issue 4 years ago • 20 comments

Hi We are running Flask service on a docker image. one of our main routes reads file from S3 and returns it's content. recently we upgraded the package from version 0.2.0 to 0.4.0.

After the upgrade, the route is working fine, but after several hours it raises strange exceptions contains only the file path (without any other information)

Our temporary solution for that is to reset the container, after that it's working fine, but fails again after several hours.

The most strange thing is that when I am connecting to the same container (with the connection issues) and I am opening python shell and execute the code - It's working as expected, even when the flask service on the same container raise exceptions on the same file.

any ideas? thanks

my code:

fs = s3fs.S3FileSystem()
with fs.open('PATH_TO_FILE.json', 'rb') as f:
    data = f.read()
    f.close()
    json_content =  json.loads(data.decode('utf-8'))

print(json_content)

ohadmata avatar Dec 02 '19 08:12 ohadmata