backoff
backoff copied to clipboard
litl
Cancel backoff-initiated sleep (for coroutines only)
When it comes time to shut down the app, I would like to get a list of all of the asyncio.Task's that are sleeping due to backoff and cancel them (causing the real exceptions - not asyncio.CancelledException - to be raised up the stacks). This is needed when the sleep is for, say, five minutes. And I want this behavior to be conditional based on 'kwargs' passed to the @ backoff 'd function. Perhaps provide access to a global list of (details, Task) tuples that can be iterated through when, say, SIGTERM is intercepted.
Ideally it would be possible to specify whether the retry could be abandoned via @backoff .. and in additional to a constant/boolean/flag, the ability to provide a function for determining whether backoff can be abandoned (pass in details so the function could use details['tries']). And a backoff.abandon() to initiate the abandonment process.
causing the real exceptions - not asyncio.CancelledException - to be raised up the stacks
The problem I see with that is that the outer task/coro would not be able to catch a CancelledError if it was swallowed by the backoff handler like this. Could you instead not catch the CancelledError that will arise from the backoff handler and inspect its cause, if it exists? If this is not possible at the moment, because the backoff handler discards the exception early, then I believe it should be changed to do that.
Edit: In fact, I just happened to look at the code and it appears my assumption was correct. If the coroutine is cancelled while it is waiting, it is still inside the exception and it will be possible to fetch the previously raised exception from the CancelledError's cause. (Assuming CancelledErrors aren't special here.) https://github.com/litl/backoff/blob/0777a0f1d184e03455a1eddf978043163f94b47d/backoff/_async.py#L94
I don’t even code in python any more
But I do! My unit tests are leaving an open socket and I think it's partially related to backoff using async code by implicit detection of asyncio.
I have two functions I want to call, where the first should be retried until successful then the second retried until successful. Currently in my unit tests it seems like both of these are being treated asynchronously.
I don't want them to stick around after my tests and I'd prefer they run synchronously in this specific use case.
