Keyboard interrupt does not work properly with ipython and TqdmCallback

[malmans2]

Hi there,

It looks like keyboard interrupt in ipython does not work properly when downloading files using TqdmCallback. I noticed it while downloading a ~1GB file:

URL = ...

import fsspec
fs = fsspec.filesystem("http", asynchronous=False)
fs.get_file(URL, "test-file", callback=fsspec.callbacks.TqdmCallback())

When I hit Ctrl+C, the download doesn't stop and continue in async mode. The traceback I get shows this:

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
Cell In[3], line 3
      1 import fsspec
      2 fs = fsspec.filesystem("http", asynchronous=False)
----> 3 fs.get_file(URL, "test-file", callback=fsspec.callbacks.TqdmCallback())

File ~/miniforge3/envs/fsspec/lib/python3.12/site-packages/fsspec/asyn.py:118, in sync_wrapper.<locals>.wrapper(*args, **kwargs)
    115 @functools.wraps(func)
    116 def wrapper(*args, **kwargs):
    117     self = obj or args[0]
--> 118     return sync(self.loop, func, *args, **kwargs)

File ~/miniforge3/envs/fsspec/lib/python3.12/site-packages/fsspec/asyn.py:91, in sync(loop, func, timeout, *args, **kwargs)
     88 asyncio.run_coroutine_threadsafe(_runner(event, coro, result, timeout), loop)
     89 while True:
     90     # this loops allows thread to get interrupted
---> 91     if event.wait(1):
     92         break
     93     if timeout is not None:

File ~/miniforge3/envs/fsspec/lib/python3.12/threading.py:655, in Event.wait(self, timeout)
    653 signaled = self._flag
    654 if not signaled:
--> 655     signaled = self._cond.wait(timeout)
    656 return signaled

File ~/miniforge3/envs/fsspec/lib/python3.12/threading.py:359, in Condition.wait(self, timeout)
    357 else:
    358     if timeout > 0:
--> 359         gotit = waiter.acquire(True, timeout)
    360     else:
    361         gotit = waiter.acquire(False)

KeyboardInterrupt:

[martindurant]

I can imagine it might be possible to cancel the coroutines when finding an interrupt, but that won't actually stop them running. It would take a exception inserted into the event-loop's thread, I think, which is also doable but less standard. I can try a couple of things.

I wonder what a minimal reproducer of this would look like, could we simulate the situation with asyncio.sleep() ?