paws icon indicating copy to clipboard operation
paws copied to clipboard

Does paws.s3_storage support http caching?

Open hadley opened this issue 4 years ago • 2 comments

When I set IfNoneMatch in get_object(), I get:

Error: SerializationError (HTTP 304). failed to read from query HTTP response body

Which suggests that this is not implemented?

hadley avatar May 18 '21 20:05 hadley

Hello! Sorry for the delayed response! Paws doesn't do any HTTP caching, at least not intentionally. I'm honestly not familiar with how one would use this option so I'm not sure what it should do in this case. What is the expected result?

Paws interprets all HTTP response codes >= 300 as errors, so it's just throwing an error. Paws is giving the wrong error message, which we need to fix. Other than the status code, S3 is returning an ordinary get_object response except that the body is empty rather than containing the object, but currently that's not available because of the error. I also tried this operation with boto3 and it returns botocore.exceptions.ClientError: An error occurred (304) when calling the GetObject operation: Not Modified.

davidkretch avatar May 20 '21 23:05 davidkretch

Yeah, this should not error; returning an empty body is what I would expect.

To use HTTP caching, you basically store the ETag and LastModified components of the response and then submit them in the request IfNoneMatch and IfModifiedSince fields. Then, if the data hasn't changed since your last request, it isn't re-downloaded and you can use what you cached.

As a general principle, I think you should only turn 400 and above into errors, not 300 and above.

hadley avatar May 21 '21 12:05 hadley