Retry on `ResponseStreamingError` when getting s3 objects.

[adriancaruana]

Describe the feature

Include ResponseStreamingError in the list of "standard" errors/exceptions that are retried when getting s3 objects.

Use Case

I'm getting ResponseStreamingError sometimes when doing .get_object with an s3 client. Even when my client is configured with retries, the client does not retry when this exception is raised.

The following code reproduces the error:

import http.server
import logging
import socketserver
import threading
from http import HTTPStatus
from io import BytesIO

import boto3
import botocore.client
from boto3.s3.transfer import TransferConfig


def repro(endpoint: str) -> None:
    session = boto3.Session(aws_access_key_id='', aws_secret_access_key='')
    client = session.client(
        "s3",
        endpoint_url=endpoint,
        config=botocore.client.Config(
            max_pool_connections=1, retries=dict(mode='standard', max_attempts=3)
        ),
    )

    client.get_object(Bucket='bucket', Key='bad')["Body"].read()


class Handler(http.server.BaseHTTPRequestHandler):
    def do_HEAD(self):
        self.send_response(HTTPStatus.OK)
        self.send_header('Content-Length', '10000')
        self.end_headers()

    def do_GET(self):
        self.send_response(HTTPStatus.OK)
        self.send_header('Content-Length', '10000')
        self.end_headers()
        # content length and body do not agree - simulating a connection drop
        self.wfile.write(b'\x00' * 1000)


if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG)

    with socketserver.TCPServer(("localhost", 0), Handler) as httpd:
        threading.Thread(target=httpd.serve_forever, daemon=True).start()
        repro(f'http://localhost:{httpd.server_address[1]}')

Proposed Solution

No response

Other Information

The following issue might be related: https://github.com/boto/botocore/issues/3132

Acknowledgements

• [ ] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

SDK version used

1.34.51

Environment details (OS name and version, etc.)

Ubuntu, python 3.11.8, urllib3 1.26.18

[tim-finnigan]

Hi @adriancaruana thanks for reaching out. In the issue you linked (https://github.com/boto/botocore/issues/3132) it looks like this was fixed via https://github.com/boto/s3transfer/pull/301 for urllib3 2+. Could you try updating your versions of Boto3/urllib3? The latest Boto3 version is 1.34.99 per the CHANGELOG. Or if you could at least try testing in 1.34.63 / urllib3 2.2.1+.

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

[weiwarren]

@tim-finnigan looks like the fix is only for s3transfer, does that apply to get_object? My understanding is get_object is not handling ResponseStreamingError explicitly?