sshfs
sshfs copied to clipboard
Problems with background threads
I am trying to use this library with FUSE. However, I have noticed:
- The FUSE "fork into background" functionality results in unkillable I/O hangs because
SSHFileSystem
seems to start background threads. I have this problem myself in rapidgzip with the thread pool, for which I added a method to join all threads, so that the process can be forked into the background, where the thread pool gets automatically restarted on first usage. It would be nice to have a similar API for SSHFilesystem. - When an error occurs, my program does not quit anymore when a SSHFilesystem was created. I have to press Ctrl+C / send SIGINT for it to quit. I guess that this is also because of background threads that do "block" somehow. Maybe I am just not closing something correctly...
The running threads according to import threading; threading.enumerate()
are: MainThread, fsspecIO, asyncio_0
. Maybe I am running into https://github.com/fsspec/sshfs/issues/42 Then again, looking at the code, it does not seem like there is any logic for joining or canceling that thread at all!?
On further inspection, the program also hangs inside sshfs.file.SSHFile.close
when calling it manually, which itself hangs in SSHFile._close, which itself hangs in fsspec.asyn.sync_wrapper, which hangs in this loop.
It would already be helpful if SSHFile.close
would forward kwargs like all the other methods that are initialized with __mirror_method
, so that an argument with timeout = 3
, for example, can be specified. Some kind of timeout should then also be used in __exit__
.