riprova icon indicating copy to clipboard operation
riprova copied to clipboard

asyncio concurrent asyncio retry does not work.

Open bt-wil opened this issue 6 years ago • 0 comments

When calling asyncio.gather with functions decorated retry, the retries would finish earlier than expected.

import asyncio
import logging

import riprova


logging.basicConfig(
    format='[%(asctime)s] %(levelname)s - %(message)s',
    datefmt="%Y-%m-%d %H:%M:%S",
    level=logging.INFO
)
LOG = logging.getLogger()


def do_retry(val):
    LOG.info("do retry %s", val)
    return True


@riprova.retry(
    backoff=riprova.ConstantBackoff(retries=5),
    error_evaluator=do_retry,
)
async def do_stuff(name):
    LOG.info('calling %s', name)
    if name == 'bar':
        await asyncio.sleep(1.0)
    else:
        await asyncio.sleep(0.1)

    raise ValueError('value-error-{}'.format(name))


async def process():
    coros = [do_stuff('foo'), do_stuff('bar')]
    # coros = [do_stuff('foo')]
    await asyncio.gather(*coros, return_exceptions=True)


loop = asyncio.get_event_loop()
loop.run_until_complete(process())
loop.close()

results:

[2018-03-29 19:17:42] INFO - calling bar
[2018-03-29 19:17:42] INFO - calling foo
[2018-03-29 19:17:42] INFO - do retry value-error-foo
[2018-03-29 19:17:42] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - do retry value-error-bar
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo

The expected result is that foo and bar should be both called 6x.

bt-wil avatar Mar 30 '18 02:03 bt-wil