Botocore mistakenly expects a reason phrase in "100 Continue" responses

[afranken]

Describe the bug

When making HTTP requests, botocore sends a 100-continue request to the server, expecting a HTTP/1.1 100 Continue response.

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

Expected Behavior

The reason phrase is optional, so a HTTP/1.1 100 response from the server is the correct behaviour. Botocore should accept a HTTP/1.1 100 response as correct.

Current Behavior

If Botocore receives a HTTP/1.1 100, it interprets the response as not correct, and does not continue with the original request.

Reproduction Steps

Start a service (locally) that sends HTTP/1.1 100 responses to 100-continue requests, e.g. S3Mock:

docker run -e initialBuckets=my-bucket --rm -it -p 9090:9090 adobe/s3mock:4.0.0

Execute a PutObject request with a random file, e.g. the README.rst from the root of this repository, e.g. through aws-cli which uses boto3/botocore internally:

aws s3api put-object \
    --bucket x \
    --key y.txt \
    --content-type text/plain \
    --endpoint http://localhost:9090 \
    --body README.rst \
    --debug

The request will fail, as the Tomcat server used in S3Mock sends a HTTP/1.1 100 as a response to the 100-continue request from botocore.

Example output:

2025-04-10 01:41:22,320 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.PutObject: calling handler <bound method UserAgentString.rebuild_and_replace_user_agent_handler of <botocore.useragent.UserAgentString object at 0x1052779b0>>
2025-04-10 01:41:22,320 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=PUT, url=http://localhost:9090/x/y.txt, headers={'Content-Type': b'text/plain', 'x-amz-sdk-checksum-algorithm': b'CRC32', 'User-Agent': b'aws-cli/2.25.13 md/awscrt#0.25.4 ua/2.1 os/macos#24.4.0 md/arch#arm64 lang/python#3.12.10 md/pyimpl#CPython m/N cfg/retry-mode#standard md/installer#source md/prompt#off md/command#s3api.put-object', 'Expect': b'100-continue', 'x-amz-checksum-crc32': b'NGSOug==', 'X-Amz-Date': b'20250409T234122Z', 'X-Amz-Content-SHA256': b'65dbc69741c062892a2842cc5d6099bbf02eefd4628d352d446c6c37cb0967da', 'Authorization': b'AWS4-HMAC-SHA256 Credential=NONE/20250409/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-checksum-crc32;x-amz-content-sha256;x-amz-date;x-amz-sdk-checksum-algorithm, Signature=e6a74e4aafab3b1c03e282cf2064d65dbb22af4396cb34460f69439d180f36d3', 'Content-Length': '42443'}>
2025-04-10 01:41:22,320 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): localhost:9090
2025-04-10 01:41:22,321 - MainThread - botocore.awsrequest - DEBUG - Waiting for 100 Continue response.
2025-04-10 01:41:22,324 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.PutObject: calling handler <function _update_status_code at 0x101d1c2c0>
2025-04-10 01:41:22,324 - MainThread - botocore.hooks - DEBUG - Event needs-retry.s3.PutObject: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x105277c80>>
2025-04-10 01:41:22,324 - MainThread - botocore.retries.standard - DEBUG - Retry needed, retrying request after delay of: 0.2322557477907542

Possible Solution

In botocore/awsrequest.py, check for only 2 parts in the response instead of 3 since the reason phrase is optional. This will let a HTTP/1.1 100 response be detected correctly.

    def _is_100_continue_status(self, maybe_status_line):
        parts = maybe_status_line.split(None, 2)

        # Check for HTTP/<version> 100 (Continue)\r\n
        return (
            len(parts) >= 2
            and parts[0].startswith(b'HTTP/')
            and parts[1] == b'100'
        )

Additional Information/Context

S3Mock is a library/service to locally test applications that integrate with AWS S3.

aws-cli version:

❯ aws --version
aws-cli/2.22.16 Python/3.12.8 Darwin/24.4.0 source/arm64

Reason phrase is optional, see https://www.rfc-editor.org/rfc/rfc9110.html#section-15.1

SDK version used

1.37.36

Environment details (OS name and version, etc.)

macOS, 15.4.1, 24E263

[adev-code]

Hello @afranken, thanks for reaching out. Looking at responses, some may have 100 Continue (2 Strings), 100-Continue (1 String) or just 100. Looking through RFCs and IETFs, it seemed that it may or may not have the word "Continue" but looking at https://datatracker.ietf.org/doc/html/rfc2616#page-48 , it's mostly 100-Continue. Although, I will present to the team if we can have:

len(parts) >= 2

on https://github.com/boto/botocore/blob/develop/botocore/awsrequest.py#L225-L232

I will let you know for any updates. Thank you.

[afranken]

@adev-code thanks for looking at this.

In RFC2616, the wording wasn't clear about whether the reason phrase is optional or mandatory, see https://datatracker.ietf.org/doc/html/rfc2616#section-6.1.1

RFC9110 is a revision of RFC2616, see https://www.rfc-editor.org/rfc/rfc9110.html#name-history-and-evolution

It has clear wording, see https://www.rfc-editor.org/rfc/rfc9110.html#section-15.1

The status codes listed below are defined in this specification. The reason phrases listed here are only recommendations -- they can be replaced by local equivalents or left out altogether without affecting the protocol.

By always expecting a reason phrase, boto3 is following RFC2616 and violating RFC9110.

[adev-code]

Hello @afranken, thanks for the patience. The change can be implemented, although we don't have a time frame on the release. Moving forward, please check our changelogs for updates (https://raw.githubusercontent.com/boto/boto3/develop/CHANGELOG.rst and https://raw.githubusercontent.com/aws/aws-cli/v2/CHANGELOG.rst) Thank you.

[afranken]

@adev-code it's been 6 weeks. Any update on the timeframe? :)

[github-actions[bot]]

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.