aiohttp_retry icon indicating copy to clipboard operation
aiohttp_retry copied to clipboard

retry_client.close() timeouts in case of exception without raise_for_status

Open Nijith1293 opened this issue 3 years ago • 1 comments

The RetryClient() object fails to close with Timeout error/Error while closing connector error when the raise_for_status is set to false

retry_options = ExponentialRetry(attempts=3, start_timeout=0.5)


async def request_start_callback(
        session: ClientSession,
        trace_config_ctx: SimpleNamespace,
        params: TraceRequestStartParams,
) -> None:
    current_attempt = trace_config_ctx.trace_request_ctx['current_attempt']
    if current_attempt > 1:
        logger.warning('"Connection Failure: Retrying %s time(s) for url %s"', current_attempt-1, params.url)


async def asyncio_retry(end_point, data, headers=None):
    trace_config = TraceConfig()
    trace_config.on_request_start.append(request_start_callback)

    retry_client = RetryClient(retry_options=retry_options, trace_configs=[trace_config])
    async with retry_client.post(end_point, data=data, headers=headers, timeout=5) as resp:
        text = await resp.text()
    logger.debug('response from the url "%s": %s', end_point, text)
    await retry_client.close()
    return text

The error observed is as below:

2022-01-28 02:19:23,726 WARNING "Connection Failure: Retrying 1 time(s) for url https://httpstat.us/503"
2022-01-28 02:19:26,576 ERROR Connection Failed
2022-01-28 02:21:35,441 ERROR Error while closing connector: ConnectionResetError(104, 'Connection reset by peer')

Nijith1293 avatar Jan 28 '22 02:01 Nijith1293

This looks like a error that's not on your end, i remember getting that and my issue was having a very short timeframe of sending the requests with that said maybe they're blocking you cause you're sending too many requests? too fast? If so try adding delay or putting it inside a try/except block with a promise to retry on fail or even execute the whole function over considering you're only doing that in the function.

outgaze avatar Apr 18 '22 02:04 outgaze

I have just tested this and have not encountered any error

inyutin avatar Aug 04 '22 13:08 inyutin

I suggest you to use last version of aiohttp_retry and last version of aiohttp

inyutin avatar Aug 04 '22 13:08 inyutin