Better support for manually-driving Session (in multiple threads)

[allisonkarlitskaya]

It's possible to construct a Session object by moving an OwnedFd, but from there your only choice is to call run() more or less. This presents a couple of issues:

• multiple threads trying to read() at the same time (acting as a form of threadpooling for requests) are not possible because run needs &mut self. Maybe you could dup() the /dev/fuse fd. I didn't try it, but it's not optimal, in any case.
• as noted in a comment in the code, the 8MB buffer is not great if we're gonna have multiple reader threads, but it's also not required for:
  • filesystems that only do writes via fd passthrough
  • read-only filesystems

So this PR is an attempt to start a discussion about maybe exposing a bit more of the internals or making them more flexible to use.

A few things that could happen to help things:

• have a mechanism to drop the buffer size. This could be done automatically for read-only filesystems. This alone would make the dup() approach less painful.
• figure out a way to make Session::run() not require a &mut self reference. This is hard because it uses that reference mostly to be able to pass mutable references to the wrapped filesystem object to the Filesystem trait impl functions, and those are highly likely to be interested in mutability.
• it's possible to use multiple Channel::receive() and the session takes an Arc on the channel. Maybe it would make sense to provide a mechanism to either accept an &Arc<Channel> when constructing the session or provide a Clone impl for Session if the underlying filesystem also allows it, taking a new ref on the channel in the process. It's quite possible to imagine a filesystem that basically puts all of its state inside of an Arc<>, using Mutex<> where necessary to maintain Sync.
• a more direct mechanism for creating Requests from a buffer and submitting them to a Filesystem impl would be interesting
• other ideas?

Thanks!

[cberner]

Can you tell me a bit more about what your high level goal? It sounds like you're trying to improve read() throughput by adding multiple threads

[rarensu]

After many months of thought, I have come to a realization. A Session does not need to own a Filesystem.

Instead of

struct Session {
    fs: Filesystem
}
impl Session {
    fn run(&mut self) {
        ...
        self.fs.methods_that_mutate();
    }
}

Why not relocate the Filesystem to be an argument of run() and friends?

struct Session {
}
impl Session {
    fn run(&self, &mut fs: Filesystem) {
        ...
        fs.methods_that_mutate();
    }
}

This solves an awful lot of my problems. Maybe it helps with yours too?