fix(s3): Force pathstyle when detecting custom endpoint url

[joe4dev]

Issue

https://github.com/aws/aws-sdk-js-v3/issues/7136

Description

S3 requests fail against emulators such as LocalStack if they use virtual-host-style S3 requests with a single endpoint for all services (e.g., AWS_ENDPOINT_URL=https://localhost.localstack.cloud:4566) rather than specifically configure a service-specific endpoint (e.g., AWS_ENDPOINT_URL_S3=https://s3.localhost.localstack.cloud:4566). This default behavior breaks seamless emulator compatibility with tools that depend on the JavaScript SDK, including the AWS Toolkit for VSCode and AWS CDK.

The Python-based botocore library solves this problem here by adjusting the addressing style (i.e., whether to force path style) before passing the options to the endpoint ruleset resolver. Therefore, Python-based AWS tools (e.g., AWS CLI, AWS SAM) support S3 requests with emulators correctly.

Similar to botocore, the proposed change adjusts the default behavior to work seamlessly with emulators to support a single endpoint URL for all services, following this AWS specification: https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html

Testing

How was this change tested?

This change solves the default behavior when using a single endpoint (i.e., AWS_ENDPOINT_URL) instead of service-specific endpoints (e.g., AWS_ENDPOINT_URL_S3) of the following cases:

• http://localhost.localstack.cloud:4566
• http://localhost:4566

Testing and comparison to botocore is done using the following reproducer: https://github.com/joe4dev/s3-endpoint-url-testing

TODO: add SDK-specific tests

Additional context

Add any other context about the PR here.

The botocore solution here is not directly portable to the JavaScript SDK due to differences in configuration:

• botocore offers addressing_style=auto|virtual|path
• aws-sdk-js offers forcePathStyle=false|true

We should maintain the option to test virtual-host-style S3 requests with emulators when providing a compatible endpoint. Therefore, we assume that a custom endpoint that includes s3. can resolve sub-domains and support virtual-host-style S3 requests.

The conditional !options.useAccelerateEndpoint attempts to mirror the botocore implementation not is_s3_accelerate_url(client_endpoint_url). The conditional !options.endpoint.includes("s3.") should rather be !options.endpoint.startsWith("s3.") because everything before s3. is treated as a bucket name in virtual-host-style requests. However, this results in a typing error because the "Property 'startsWith' does not exist on type 'Provider'.".

Should such a change be part of the Smithy rules engine to be applicable for all AWS SDKs? Example from botocore for S3 endpoint ruleset: https://github.com/boto/botocore/blob/d1fd992119b5df4f4d2169e2383ab99288466e8b/botocore/data/s3/2006-03-01/endpoint-rule-set-1.json

Checklist

• [ ] If you wrote E2E tests, are they resilient to concurrent I/O?
• [ ] If adding new public functions, did you add the @public tag and enable doc generation on the package?

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: I work for LocalStack