Request doesn't provide a way to refresh headers for each retry, making it impossible to regenerate expired Authorization headers

[vitaly-krugl]

I am using requests 2.23.0 with Python 3.8.

My app configures the requests.packages.urllib3.util.retry.Retry instance and then passes it via the max_retries arg to requests.adapters.HTTPAdapter constructor and then mounts the adapter as follows:

class RetriableSslSession(requests.Session):
    def __init__(self):
        super(RetriableSslSession, self).__init__()

        self.mount('https://', create_default_ssl_adapter())

Then before making a request, my app sets session.auth to an instance of EdgeGridAuth that is derived from requests.auth.AuthBase.

Since EdgeGridAuth implements the AuthBase.__call__ method, I expected that the Authorization header would be generated for each retry. However, in our apache proxy logs, we're seeing the same Authorization header reused from time to time, which suggests that retries are not causing the Authorization header to be generated.

Could you please let me know what I may be doing wrong and explain how to cause the Authorization header to be re-generated for each Retry?

[nateprewitt]

Hi @vitaly-krugl, to clarify your issue, you're seeing the same Authorization reused in all retries or is the issue intermittent?

If it's consistent, can you provide a simple reproduction using only code from Requests and urllib3? That will help rule out issues in your Auth, Session, and Retry implementations. If it's intermittent, we'll need more info around the retries and what triggered them. That will help identify if this is in fact a bug in Requests.

[vitaly-krugl]

@nateprewitt - per your understanding, is the auth hook (AuthBase.__call__()) supposed to be called for each retry attempt?

[nateprewitt]

I believe your auth hook should only be invoked here which happens during request preparation unless otherwise invoked directly.

If your Adapter isn't handling anything with retries and relies solely on urllib3, I wouldn't expect auth to be recomputed.

[sigmavirus24]

If your Adapter isn't handling anything with retries and relies solely on urllib3, I wouldn't expect auth to be recomputed.

This is 100% correct. You're operating at 2 different levels

[vitaly-krugl]

Thank you all for confirming that my current retry configuration approach doesn't facilitate re-computation of auth upon each retry. In this case, what is the correct way to configure retries and auth so that auth will be recomputed upon each retry?

Known working doc links/example links would be much appreciated! Thank you in advance.

[vitaly-krugl]

I also added the comment in the urllib3 issue asking for additional hooks in urllib3 that would enable python-requests to add support for updating auth during retries.

[sigmavirus24]

Without knowing what you're telling urllib3 to retry or what the exact behavior is of your service we can't help you and support requests are best asked on Stack overflow.

[vitaly-krugl]

(cc @sigmavirus24)

I need the AuthBase.__call__ method of the EdgeGridAuth instance which I assign to session.auth to be called for each retry attempt that is attempted as the result of the urllib3 Retry instance that I assigned as outlined in the description section of this issue in order to re-generate the Authorization header.

The reason for this requirement is that the Authorization header value has a short TTL, so it needs to be regenerated for each retry attempt and the new value of the Authorization header needs to be used by urllib3 when it's sending the corresponding retry request to the server.

The Retry instance is configured as follows:

urllib3_retry.Retry(
    total=DEFAULT_MAX_TOTAL_RETRIES,
    connect=DEFAULT_MAX_CONNECTION_RETRIES,
    read=DEFAULT_MAX_READ_RETRIES,
    redirect=0,
    method_whitelist= frozenset(
        {'PUT', 'GET', 'OPTIONS', 'HEAD', 'DELETE', 'TRACE'}),
    status_forcelist= frozenset(
        {429, 500, 501, 502, 503, 504, 505, 506, 507, 509, 510, 598}),
    backoff_factor=1/10000,
    raise_on_redirect=False,
    raise_on_status=False)

[keyvanm]

I have the same problem. I'm hitting an API that requires an Auth-Signature header, which is a signed hexdigest using a nonce based on the UNIX value of now. In the last retry, the signature is already expired. It would be great if there was a way to recalculate a header value on each retry.

[vitaly-krugl]

Dear maintainers, is there a solution in the works for this?