httpio icon indicating copy to clipboard operation
httpio copied to clipboard

Add fallback to GET if HEAD doesn't work

Open samdbmg opened this issue 4 years ago • 0 comments

This works around a problems where servers don't support HEAD correctly, such as AWS S3 presigned URLs which incorporate the method into the URL signature, so you can't produce one that supports both GET and HEAD. This PR adds a fallback behaviour in response to 405 Method Not Implemented or 403 Forbidden (S3's response) on HEAD to try a GET instead, but avoids reading the response body to prompt the server not to send any more data.

I've also made a tweak to the unit tests in the process to generate data with the slightly faster (but less random) random.getrandbits and factor out sample data generation across both tests.

cc @jamesba - would you mind reviewing this too?

samdbmg avatar Jul 28 '21 16:07 samdbmg