where to put my modular filesystem components?

[rarensu]

I have created a toolbox of modular filesystem components. My hope is that this toolbox will be maintained by the community, so that each component will become optimized, and the toolbox will be extended over time.

Code examples gist

I initialized the toolbox with seven components.

• BranchFS contains multiple inner filesystems
• BindFS redirects to an external, underlying filesystem (with kernel passthrough!)
• CacheFS stores recently used data in memory
• EncryptFS for simple password encryption of regular files
• MonitorFS for extra logging
• EmptyFS is a placeholder inner filesystem for development
• AssertFS for unit testing. I used it to write unit tests for several of the above! See gist.

I envision that a toolbox of components will encourage more people to adopt FUSE. The barrier to entry is lower if a new user can assemble the Filesystem they want out of pre-made components. For example, I believe the majority of FUSE Filesystem concepts can be described as a layer on top of BindFS. But kernel passthrough is really quite confusing, so users probably feel apprehension about getting started. I hypothesize that a pre-made BindFS component with kernel passthrough will help bring many new FUSE developers to the scene.

To demonstrate the convenience of assembling a filesystem from components, I created an example application I named the six-layer-filesystem (all except for AssertFS). See gist.

    1  BranchFS 
    2  ├─ MonitorFS
    3  │  └─ CacheFS
    4  │     └─ EncryptFS
    5  │        └─ BindFS
    6  └─ EmptyFS

I also introduce a new convenience called FilesystemLayer, which describes a Filesystem that wraps around an inner Filesystem. FilesystemLayer is a trait that has the same methods as the Filesystem trait (and in fact, implements the Filesystem trait); they are dyn compatible. The main difference is that default implementations of FilesystemLayer redirect to the inner Filesystem instead of returning an error. This makes it very easy to create new Filesystem components by just implementing a handful of methods. For example, EncryptFS (above) is a FilesystemLayer that literally only implements two methods: read() and write()!

Question: should I integrate the toolbox into Fuser crate, or segregate it as a separate crate?

Reasons to integrate:

• AssertFS in particular is so useful for writing unit tests that I would immediately use it to write unit tests for the Fuser library core itself (for example, unit testing the dispatch() function).
• The Fuser crate and the toolbox are guaranteed always compatible.

Reasons to segregate:

• As the toolbox gains components, it could weigh down the Fuser crate and become a maintenance burden.
• The toolbox could add support for Fuse libraries other than Fuser.

[cberner]

I think a separate crate makes the most sense. That way you'll be able to develop it without blocking on me reviewing PRs (I don't have much time to spend on fuser these days)