cberner
Manually instantiate test request and reply without session
Howdy,
I want to be able to call my filesystem's functions but have the reply come back to me instead of the kernel. No "session" should be needed.
I have two potential use cases.
- Unit testing.
- Nesting fuser filesystems within other fuser filesystems.
I expect that any kind of solution would likely involve changing the core trait function signatures and/or exposing additional interfaces as public.
Like this, maybe:
fs=My_FS::new();
(fake_sender, fake_receiver)=fuser::FakeSenderReceiver::new();
test_request=fuser::Request::new();
test_replyentry=fuser::ReplyEntry::new(1, fake_sender);
fs.lookup(&test_request, 1, "foo.txt", test_replyentry).unwrap();
println!("{:?}", fake_receiver.attr);
Or, like this maybe:
fs=My_FS::new();
test_request=fuser::Request::new();
mut test_replyentry=fuser::FakeReplyEntry::new(1, None);
fs.lookup(&test_request, 1, "foo.txt", &mut test_replyentry).unwrap();
println!("{:?}", test_replyentry.attr);
Please share your thoughts before I fork and hack. I want to maximize the chances of a contribution.
Hmm, I haven't thought much about the use case of user unittesting, but I think the best interface would be something like:
fn mount_test<FS: Filesystem>(fs: FS) -> MockKernel
and then MockKernel should be an object with methods that correspond to each syscall, and it would call the FS and have some way to retrieve / assert the expected result and return a reply to the FS.
Nesting fuser filesystems within other fuser filesystems
What do you mean by this?
I wanted the mock kernel to have a maximally similar interface as the filesystem interface, so the user doesn't have to learn anything new.
Like this
struct MockKernel{
fs: <Filesystem>
}
impl MockKernel{
fn lookup(args...)->Result(){
//(set up request, reply)
self.fs.lookup(request, args..., reply);
//(retrieve response)
return Ok((attr, ...))
}
}
apologies for incorrect annotation syntax.
I suppose it's not a mock kernel; it doesn't have notifications, for example. I would call it an internal interface to a filesystem.
So I suppose the public function might be something like
fn mount_internal<FS: Filesystem>(fs: FS) -> InternalFilesystem
Hmm, that doesn't sound like the right approach to me. But taking a step back, can you tell me more about the tests you're trying to write?
The way that I test my fuser filesystems is that I mount them, and then use standard tools like "ls", "touch", "mkdir"...etc and check the results with those same tools. Admittedly, that's more of an integration test, but I've found it to work well
What you have described is exactly an integration test. I already have those. They are also important.
I want to combine multiple fuse filesystems to avoid lots of useless context switching.
Example. Three fuse filesystems. Lots of useless context switching.
flowchart TD
A[Kernel] -->|Request| B(Branch FS)
B --> C{logic}
C -->|Passthrough| D[Kernel]
C -->|Passthrough| E[Kernel]
D -->|Request| F(SSH FS)
E -->|Request| G(Squash FS)
G --> I[local file]
F --> J[remote host]
Better
flowchart TD
A[Kernel] -->|Request| B(Branch FS)
B --> C{logic}
C -->|Internal call| F(SSH FS)
C -->|Internal call| G(Squash FS)
G --> I[local file]
F --> J[remote host]
Ah, I see, thanks. It looks like you want a more generic filesystem library that will allow composing different implementations. The fuser library is effectively a Rust rewrite of libfuse and I don't think it's a good place for higher level functionality like that. What I'd suggest doing is make your own crate with a new e.g. ComposableFilesystem that has the interface you want, and then add an impl Filesystem for ComposableFilesystem. That way you can still use fuser to mount the filesystem and handle kernel interactions, but you'll also be able to combine them like you're describing and test them
If i only wanted to be able to internally interface with my own filesystems, then that would be acceptable. I was hoping I could also compose fuser filesystems from other people's crates. If fuser crates could natively get along, it might be very healthy for the community. But even if you don't philosphically agree with moving in that direction, maybe you be willing to support unit testing? Surely that is a worthy cause.
It's a very small ask. I don't think I need to change the fuser filesystem trait or even add a new feature. I just need the compiler to permit me to implement one of the internal fuser traits; it currently won't because they are not marked as public. A single well-placed pub would literally be enough! Then we get unit testing and/or composability. Why not let me prove it to you?
I'm always open to looking at PRs. I'm just setting expectations that I will likely reject it.
On Wed, Apr 30, 2025 at 11:54 AM Richard Lawrence @.***> wrote:
rarensu left a comment (cberner/fuser#335) https://github.com/cberner/fuser/issues/335#issuecomment-2842983114
If i only wanted to be able to internally interface with my own filesystems, then that would be acceptable. I was hoping I could also compose fuser filesystems from other people's crates. If fuser crates could natively get along, it might be very healthy for the community. But even if you don't philosphically agree with moving in that direction, maybe you be willing to support unit testing? Surely that is a worthy cause.
It's a very small ask. I don't think I need to change the fuser filesystem trait or even add a new feature. I just need the compiler to permit me to implement one of the internal fuser traits; it currently won't because they are not marked as public. A single well-placed pub would literally be enough! Then we get unit testing and/or composability. Why not let me prove it to you?
— Reply to this email directly, view it on GitHub https://github.com/cberner/fuser/issues/335#issuecomment-2842983114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGNXQC2YRIHTPTSVIKHSMD24EL4VAVCNFSM6AAAAAB4F5SKZ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDQNBSHE4DGMJRGQ . You are receiving this because you commented.Message ID: @.***>
The other approach that might work would be https://github.com/cberner/fuser/issues/164
I haven't had time to work on it, but my idea was to create a new async friendly interface that also removed the usage of Reply in the public API. That would probably enable the same composability that you're looking for
Oh actually, have you looked at Session::from_fd? I haven't tried using that for unit testing, but it seems like you might be able to pass in an fd that you control instead of a /dev/fuse fd
After some thinking, I have realized that while it is theoretically possible to achieve my goal just by adding some pub statements to the fuser crate, the resulting code on my end would be horrifically complicated and not much better than the file descriptor option fuser already has. That's a lot of work for very minimal benefit while also needlessly complicating the crate api for the people who didn't care. I don't think that path is worth it for me to pursue at this time. However, I still feel it's worthwhile to take a closer look at the paths that do involve some change to the features of the fuser crate.
For example, suppose that the fuser crate contained an alternative implementation of request and reply that didn't use channels and senders at all. Instead, they would pass the data to some kind of sharable memory structure. You could hide that implementation behind configuration feature. Users who didn't care would never turn that feature on and for them the api is unchanged. I could turn the feature on and build cool new things. Fuser already has some features like that, for example, the notifier doesn't work unless you turn on a recent abi. So I wonder if this path is more amenable to you.
Another way of looking at it this way is that putting the reply into a sharable memory structure is something that multithreading can understand. Lots of worker threads can process the filesystem methods In parallel and save the replies for later. a session thread eventually reads their replies and passes them along to the kernel. A side effect of doing it this way is that I can then more easily disconnect the session loop and replace it with something else. So my desire to do unit testing and my desire to do multithreading can be best friends.
As demonstrated, I can be wrong. Maybe I am wrong again. But I'm not going away. This project is important to me. I appreciate that you spend the time to read my posts. If you can tolerate my janky personality, I can promise there will be contributions.
I did a re-write in my fork so that the ReplyX are traits instead of structs, and Reply is a struct instead of a trait.
pub struct Reply {
unique: ll::RequestId,
sender: Option<Box<dyn ReplySender>>,
}
impl Reply {
pub fn new<S: ReplySender>(unique: u64, sender: S) -> Reply { ... }
fn send_ll_mut(&mut self, response: &ll::Response<'_>) { ... }
fn send_ll(mut self, response: &ll::Response<'_>) { ... }
fn error(self, err: c_int) { ... }
}
pub trait ReplyError { ... }
pub trait ReplyEntry: ReplyError { ... }
impl ReplyEntry for Reply {
fn entry(self, ttl: &Duration, attr: &FileAttr, generation: u64) {
self.send_ll(&ll::Response::new_entry( ... ));
}
}
pub trait Filesystem {
fn lookup(&mut self, req: &Request, parent: u64, name: &OsStr, reply: impl ReplyEntry) { ... }
}
impl Filesystem for My_FS{
fn lookup(&mut self, req: &Request, parent: u64, name: &OsStr, reply: impl ReplyEntry) {
reply.entry( ... )
reply.error( ... )
}
}
In my opinion, this is the more logical way to express the meaning of the ReplyX type. It is a type with a .x() method and a .error() method. That is exactly what traits are for.
So does it work? Yes. Screenshot. It works. It should in theory yield the same executable. I didn't add a Box