FUSE Passthrough Support

[darthShadow]

Merged for the 6.9 Kernel: https://www.phoronix.com/news/Linux-6.9-FUSE-Passthrough

Relevant Git Patch: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6ce8b2ce0d7e3a621cdc9eb66d74436ca7d0e66e

[half-life666]

Is there any update on this? I'm interested in this passthrough feature, and looking at libfuse support here on passthrough, seems not too complicated.

I can have a look at how to implement this, but I do not have a good understanding on the code structures of go-fuse, e.g., what is the equivalent implementation of struct fuse_file_info in libfuse?

[hanwen]

see https://review.gerrithub.io/c/hanwen/go-fuse/+/1199984

unfortunately, it looks buggy - the newly added benchmark fails if there are multiple processes accessing the same file. Will need more scrutiny.

[hanwen]

working now. The speed up is considerable (7x!) and makes the whole business of splicing obsolete.

[darthShadow]

Looking through the changes, would it be possible to enhance the PassthroughFd function to also return an error (and consequently short-circuit the backing id function) along with the fd?

I’m mostly interested in this for rclone where we support both local storage (which will have an fd) & cloud storage (which won't have one) or even a combination of the two using the Union backend.

[hanwen]

short-circuit the backing id function

by short-circuit, you mean disable the passthrough feature?

Good point.

[darthShadow]

Yeah, I was thinking of disabling it altogether when it's purely cloud storage and just skip it for that file for a mixed cloud+local remote and the file is cloud-backed only.

[hanwen]

see https://review.gerrithub.io/c/hanwen/go-fuse/+/1199984/10..11

[darthShadow]

Looks good, but I was thinking along the lines of returning a syscall.Errno like the other interfaces for a file:

• syscall.ENOSYS for a purely cloud-storage remote where no file will ever have an fd so the feature can be disabled outright.
• syscall.EINVAL for a mixed remote where the backing file is a cloud-storage file.
• 0 for a purely local remote or if the backing file is a local file for a mixed remote.

[rfjakob]

Instead of ENOSYS you can disable it in the mountoptions, no?

On Tue, 27 Aug 2024, 12:52 Anagh Kumar Baranwal, @.***> wrote:

Looks good, but I was thinking along the lines of returning a syscall.Errno like the other interfaces for a file:

• syscall.ENOSYS for a purely cloud-storage remote where no file will ever have an fd so the feature can be disabled outright.
• syscall.EINVAL for a mixed remote where the backing file is a cloud-storage file.
• 0 for a purely local remote or if the backing file is a local file for a mixed remote.

— Reply to this email directly, view it on GitHub https://github.com/hanwen/go-fuse/issues/508#issuecomment-2312193852, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACGA7ZM5TW5DP37GYGGKNTZTRK73AVCNFSM6AAAAABEZJU5ESVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJSGE4TGOBVGI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[hanwen]

For the "disable outright", a config/mount option is the right approach (we don't expose it, though), but the check is cheap, so there not much benefit in providing a "disable outright" knob.

Having an errno will require more documentation (possible values) and raises the question what to do if the errno is nonzero, but not one of the options mentioned above.

[darthShadow]

Yeah, a mount option, if added, would work just as well.

But since these will be called only on Open/Create, a boolean check on every file should be plenty fast. Anything additional will just be minor improvements.

[hanwen]

implemented in https://github.com/hanwen/go-fuse/commit/e0a0b09ae8287249c38033a27fd69a3593c7e235