tenacity icon indicating copy to clipboard operation
tenacity copied to clipboard

Better API for coroutine-returning function?

Open betafcc opened this issue 7 years ago • 1 comments

First of all, awesome package, first day using it and it's already useful Second, it's my first day using it, apologies if I'm not using it as intended, but:

It is common to wrap coroutine-functions (ie, async/await functions) in normal functions for argument handling or do misc decorators or higher-order-functions stuff, one realistic example:

from functools import partial
import aiohttp
from tenacity import retry

# Normal coroutine function
async def fetch(method, url, *args, **kwargs):
    print('Will try to ', method, url)
    async with aiohttp.ClientSession() as session:
        async with session.request(method, url, *args, **kwargs) as response:
            return await response.text()

# wrapping with normal function 
get = partial(fetch, 'get')
# or
get = lambda *args, **kwargs: fetch('get', *args, **kwargs)
# or even
def get(*args, **kwargs):
    return fetch('get', *args, **kwargs)

So, the following work as expected: asyncio.run(retry(fetch)('get', 'https://www.foo.bar')) And retries forever, but not the following: asyncio.run(retry(get)('https://www.foo.bar')), in any of the 3 forms defined for get

That makes sense, the function is interpreted as a normal function by retry, one which successfully returns a coroutine. I could instead do async def get(*args, **kwargs): return await fetch('get', *args, **kwargs) and it would work, but that it's just not practical, the pattern of wrapping or partial applying or decorating coroutine functions probably should be covered by tenacity.

Reading the code I noticed I can use AsyncRetrying().wraps(get)('https://www.foo.bar') and it would work as expected, maybe this should be documented? Or maybe a better api (say retry_awaitable) for this common case?

betafcc avatar Jul 30 '18 13:07 betafcc

You should think long and hard before trying to detect async functions since it is impossible to avoid false negatives.

https://github.com/python-trio/trio/issues/635

https://github.com/vxgmichel/aiostream/issues/53

I think it is better to be explicit.

So either retry_awaitable

or from retry.async import retry - for an async version of retry.

andersea avatar Feb 21 '20 16:02 andersea