Endpoint resolution & access points

[jamesbornholt]

Currently our endpoint resolution logic is the very naive:

format!("{}.s3.{}.amazonaws.com", bucket, self.region);

But this doesn't support:

• Dual-stack endpoints (IPv6)
• FIPS endpoints
• Transfer acceleration endpoints
• Regions that don't end in amazonaws.com (e.g., China regions)
• Access points
• PrivateLink endpoints

It doesn't look like the CRT has any built-in support for dealing with these, so we might need to do it ourselves.

[monthonk]

Can we reuse some code from aws sdk rust? I see they have something like endpoint_resolver() in their code, so we don't have to implement it ourselves.

[jamesbornholt]

Oh neat! That will save us from dealing with the other partitions, at least. We still need to figure out access points.

[jamesbornholt]

Thinking about it a bit more, there's roughly three things we need to do here:

1. Correct endpoint resolution for regions outside the aws partition
2. Allow the customer to provide an endpoint URL like the SDKs do—this should enable FIPS, IPv6, Transfer Acceleration, PrivateLink, and Multi-Region Access Points (and we should test these)
3. Document that (single-region) Access Points can be used via access point aliases

[jamesbornholt]

Also need to test S3 on Outposts here.

[jamesbornholt]

#61 will let us specify custom endpoint URLs. But some of these things work without that:

• Access points can be specified using bucket aliases
• Gateway VPC endpoints are a networking-level thing and should just work

I've tested both these and they are working on mainline.

[dannycjones]

There's a little bit of experimental code on this branch which may be good to use as a reference point: https://github.com/awslabs/mountpoint-s3/tree/endpoint-rule-engine

[jamesbornholt]

#428 wrapped this one up, except for testing, which is in #417.