distributed icon indicating copy to clipboard operation
distributed copied to clipboard

`Future.__del__` logs ignored exceptions during interpreter shutdown

Open hendrikmakait opened this issue 2 years ago • 1 comments

While I don't have a reproducer, multiple users have reported seeing many logs about Future.__del__ ignoring exceptions during interpreter shutdown. #8449 was supposed to address this, but it failed to do so:

Before #8449

Exception ignored in: <function Future.__del__ at 0x...>
Traceback (most recent call last):
  File "/python/lib/python3.10/site-packages/distributed/client.py", line 510, in __del__
AttributeError: 'NoneType' object has no attribute 'is_finalizing'

After #8449

Exception ignored in: <function Future.__del__ at 0x...>
Traceback (most recent call last):
  File "/python/lib/python3.10/site-packages/distributed/client.py", line 510, in __del__
TypeError: 'NoneType' object is not callable

From what I understand, this fails because of this caveat: https://github.com/python/cpython/blob/e39ae6bef2c357a88e232dcab2e4b4c0f367544b/Doc/c-api/init.rst#L403-L405

Empirically, we can fix the Future.__del__ problem by binding sys.is_finalizing to a class variable which will avoid its destruction. However, I assume that is_python_shutting_down has a similar problem. While this issue isn't urgent, the chattiness is annoying and might distract from genuine issues on shutdown.

hendrikmakait avatar Feb 07 '24 18:02 hendrikmakait

Since this is otherwise a benign error, would there be any harm in rewriting the test as if (is_python_shutting_down is not None and not is_python_shutting_down())?

csubich avatar Oct 23 '24 16:10 csubich