cberner
WIP: Add Multithreading Support
Add Multithreading Support to fuser.
This implementation modifies BackgroundSession to accept a thread quantity. If this number is two or more, then n - 1 background workers will be created. There will always be a primary worker created, hence n - 1.
Session::worker is called to clone the Session, which in turn calls Channel::worker. This method accepts the Mount object, and using .session_fd to return the Session FD, fuse_fd_clone is invoked to clone the file descriptor and run the needed ioctl call to setup the session.
I can't take all the credit, as the main piece of code in channel.rs is from the Datenlord project where they are doing a native async_fuse library.
The code in question is from here: https://github.com/datenlord/datenlord/blob/d90fd43732373451207e56e9b9cd7eef9e7b53e1/src/async_fuse/fuse/session.rs#L73
Additional References:
- https://john-millikin.com/the-fuse-protocol#multi-threading
- https://fuse-devel.narkive.com/CodQceXu/how-could-fuse-support-parallel-opreation
TODO
- [ ] Clean up feature gate
- [ ] Add more function documentation
- [x] Add changelog entry
- [x] Bump version
Do mind explaining why you picked the multithreading programming model rather than async/await programming model?
Do mind explaining why you picked the multithreading programming model rather than async/await programming model?
Because that would be a major refactor of the code base, and to do it properly everything would need to be rewritten
@volfco I created a similar multithreading solution but instead of making a whole new session object for each worker, I just created a new channel object for each worker, and one session holds an array of channels. The entire session goes into an arc, rather than each of its mutable fields. Then the run function becomes an associated function that takes Arc<Session> rather than a traditional method that takes Self (Of course, just putting the Filesystem in an Arc would have accomplished the same thing. So maybe I over-engineered it? Oh well). This worked for me because at the same time, I was also changing the method signatures of Filesystem to take &self rather than &mut self, which is very nearly a requirement of async. This combination puts less pressure on Filesystem to behave a certain way, because there is no longer any need to clone Filesystem. It just needs to be Send + Sync. If you're still working on this, we can compare notes. For example, my strategy has far fewer unsafe blocks than yours (really, just the fuse worker ioctl function). I'm not really sure why yours has so many.
@rarensu I'm not currently working in this space and on this project. At this point, I'm waiting for more robust Rust support for the kernel and then I'll reimplement my idea as a kernel module.
Even with multi-threading, the overhead was still quite high. Maybe I'll revisit this eventually