charliecloud
charliecloud copied to clipboard
hpc
SquashFS: use multi-threaded FUSE loop
Currently we use a single-threaded FUSE loop (fuse_session_loop()) rather than a multi-threaded loop (fuse_session_loop_mt()).
This follows SquashFUSE, which is also single-threaded and has a brief FIXME about multi-threading. We should probably talk with the SquashFUSE people before changing to multi-threaded because there may be good reasons SquashFUSE is not ready for multi-threading.
I'm guessing that multi-threading will improve performance, though IIRC @shanegoff and @mphinney1100's testing did not turn up problems when using the single-threaded SquashFUSE squashfuse_ll executable. So I guess some performance evaluations are required before committing (no pun intended).
Squashfuse might actually automatically pick multi threading based on the output of the help text.
(base) $ squashfuse_ll --h
squashfuse 0.1.103 (c) 2012 Dave Vasilevsky
Usage: squashfuse_ll [options] ARCHIVE MOUNTPOINT
FUSE options:
-d -o debug enable debug output (implies -f)
-f foreground operation
-s disable multi-threaded operation
There is a flag to disable multi threaded operation, so I would expect we're actually already using multi threaded operations where it decides to.
All I can find in the code is the single-threaded operation. This seems to be the crux of it (linked above):
/* FIXME: multithreading */
err = fuse_session_loop(ch.session);
If I missed something please LMK!