Proxy configuration is not respected when fetching credentials from IMDS endpoint

[alexmv]

Describe the bug The Config object supports passing in a proxies value, which can be used to override the proxies taken from the environment. However, this appears to only be used for requests to the S3 endpoints themselves -- it is not used for IMDS requests.

Steps to reproduce When fetching credentials via the IAM role on an EC2 instance, the proxy configuration is not used.

Expected behavior This does not use the configured proxy:

import logging

import boto3
from botocore.client import Config

boto3.set_stream_logger("", logging.DEBUG)
session = boto3.Session()
client = session.resource(
    "s3",
    config=Config(proxies={"http": "http://localhost:9999", "https": "http://localhost:9999"}),
)

Here's what it would look like if it used the proxy:

2022-03-23 23:04:57,380 botocore.credentials [DEBUG] Looking for credentials via: iam-role
2022-03-23 23:04:57,380 urllib3.connectionpool [DEBUG] Starting new HTTP connection (1): localhost:9999
2022-03-23 23:04:57,418 urllib3.connectionpool [DEBUG] http://localhost:9999 "PUT http://169.254.169.254/latest/api/token HTTP/1.1" 200 56
2022-03-23 23:04:57,419 urllib3.connectionpool [DEBUG] http://localhost:9999 "GET http://169.254.169.254/latest/meta-data/iam/security-credentials/ HTTP/1.1" 200 19

Instead:

2022-03-23 23:16:50,729 botocore.credentials [DEBUG] Looking for credentials via: iam-role
2022-03-23 23:16:50,729 urllib3.connectionpool [DEBUG] Starting new HTTP connection (1): 169.254.169.254:80
2022-03-23 23:16:51,732 botocore.utils [DEBUG] Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/api/token: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"

The proxy configuration looks like it needs to be plumbed through to here: https://github.com/boto/botocore/blob/a9142888a84c3e6d32ea8eb30decfe64eee6d729/botocore/utils.py#L393-L414

[tim-finnigan]

Hi @alexmv thanks for reaching out. I’m going to bring this up for discussion with the team and we will post an update here.

[alexmv]

Thanks for the update!

[tim-finnigan]

Linking related issue that provides more context on problems with the proxy configuration: https://github.com/boto/botocore/issues/2707.