S3Client instantiation extremely slow

[pitrou]

Describe the bug

One of our users has reported that creating a S3Client instance can take up to 1 millisecond. According to profiler statistics, most of the time is spent in Aws::Crt::Endpoints::RuleEngine (see linked issue).

Expected Behavior

I would expect a S3Client to be reasonably fast to instantiate (less than a microsecond).

Current Behavior

See description and profile graph excerpt in linked issue.

Reproduction Steps

Using PyArrow:

>>> from pyarrow.fs import S3FileSystem
>>> %timeit s = S3FileSystem()
1.29 ms ± 1.27 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
>>> %timeit s = S3FileSystem(anonymous=True)
1.29 ms ± 5.52 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
>>> %timeit s = S3FileSystem(anonymous=True, region='eu-west-1')
1.29 ms ± 4.86 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.267

Compiler and Version used

gcc 12.3.0

Operating System and version

Ubuntu 22.04