tenacity
tenacity copied to clipboard
Better API for coroutine-returning function?
First of all, awesome package, first day using it and it's already useful Second, it's my first day using it, apologies if I'm not using it as intended, but:
It is common to wrap coroutine-functions (ie, async/await functions) in normal functions for argument handling or do misc decorators or higher-order-functions stuff, one realistic example:
from functools import partial
import aiohttp
from tenacity import retry
# Normal coroutine function
async def fetch(method, url, *args, **kwargs):
print('Will try to ', method, url)
async with aiohttp.ClientSession() as session:
async with session.request(method, url, *args, **kwargs) as response:
return await response.text()
# wrapping with normal function
get = partial(fetch, 'get')
# or
get = lambda *args, **kwargs: fetch('get', *args, **kwargs)
# or even
def get(*args, **kwargs):
return fetch('get', *args, **kwargs)
So, the following work as expected: asyncio.run(retry(fetch)('get', 'https://www.foo.bar'))
And retries forever, but not the following: asyncio.run(retry(get)('https://www.foo.bar')), in any of the 3 forms defined for get
That makes sense, the function is interpreted as a normal function by retry, one which successfully returns a coroutine.
I could instead do async def get(*args, **kwargs): return await fetch('get', *args, **kwargs) and it would work, but that it's just not practical, the pattern of wrapping or partial applying or decorating coroutine functions probably should be covered by tenacity.
Reading the code I noticed I can use AsyncRetrying().wraps(get)('https://www.foo.bar') and it would work as expected, maybe this should be documented? Or maybe a better api (say retry_awaitable) for this common case?
You should think long and hard before trying to detect async functions since it is impossible to avoid false negatives.
https://github.com/python-trio/trio/issues/635
https://github.com/vxgmichel/aiostream/issues/53
I think it is better to be explicit.
So either retry_awaitable
or from retry.async import retry - for an async version of retry.