mergerfs
mergerfs copied to clipboard
trapexit
setcap cap_dac_read_search=+ep doesn't work
Describe the bug
setcap cap_dac_read_search=+ep doesn't allow binaries to bypass permissions for files mounted through mergerfs.
To Reproduce
$ echo "foo" > /mnt/storage/bar
$ chmod 000 /mnt/storage/bar
$ cp `which cat` ~/
$ sudo setcap cap_dac_read_search=+ep ~/cat
$ cat /mnt/disk2/bar
cat: /mnt/disk2/bar: Permission denied
$ ./cat /mnt/disk2/bar
foo
$ ./cat /mnt/storage/bar
./cat: /mnt/storage/bar: Permission denied
Expected behavior
$ ./cat /mnt/storage/bar
foo
System information:
- OS, kernel version:
uname -a:Linux 5.15.53 #1-NixOS SMP Thu Jul 7 15:53:35 UTC 2022 x86_64 GNU/Linux - mergerfs version:
mergerfs -V:mergerfs version: 2.33.3 - mergerfs settings
/mnt/disk* /mnt/storage fuse.mergerfs cache.files=partial,use_ino,dropcacheonclose=true,defaults,allow_other,minfreespace=50G,fsname=mergerfs,posix_acl=true 0 2 - A strace of the application having a problem:
-
strace -fvTtt -s 256 -o /tmp/app.strace.txt <cmd>app.strace.txt
-
- strace of mergerfs while app tried to do it's thing:
-
strace -fvTtt -s 256 -p <mergerfsPID> -o /tmp/mergerfs.strace.txtmergerfs.strace.txt
-
cat isn't the app which is access the file. mergerfs is. You'd have to give that cap to mergerfs. mergerfs doesn't handle permissions. The kernel does. So I think when you set cat to have that cap the kernel sends the command to mergerfs and since you don't have the cap on mergerfs and is executing the request as uid/gid 1000 because that's who issued the request it fails.
I'm not sure it's ideal to have mergerfs with such cap but caps aren't sent along with requests so the best I could do is attempt to query the caps on the fly and that's really not ideal.
I still find it surprising. Compare it to user / group permissions:
- They are also enforced by the kernel (I assume).
- mergerfs runs as root, not as the user running cat.
- mergerfs passes on the uid / gid of the user running cat.
As a user, I would expect similar functionality for cap based permissions. That said, I can understand if it isn't so easy. The discussion in the mergerfs docs about security_capability and xattr made me think I might be missing something. If you want to resolve as WAI, I understand.
What's surprising? Caps aren't part of the FUSE protocol. There are lots of extra kernel features that aren't. Namespaces for instance. There isn't anything in the protocol that shares the caps of the executable making the request. Just pid, uid, gid. security_capability and xattr are different. xattr is a specific call. The kernel issues a getxattr and then manages the permissions just as it does with regular permissions (issues a getattr and responds to those values prior to mergerfs ever seeing anything.) There is no "getcap" request in FUSE. There is no set of caps in the requests. It doesn't matter if mergerfs is running as root. The request only contains the information that the request comes from process with (pid,uid,gid). So the best I could do is on every single request look up the process, look up the executable, make a request of caps, change caps on the executing thread, do request, change back. And since process IDs constantly change it couldn't really be cached without being a safety risk. It's not about "easy" or not. It's about the possible overhead and security risks.
I can look into it just understand there will be an added overhead of at least 2 syscalls to every relevant request. It might be worth checking if just setting said cap on mergerfs is sufficient as that should be without the overhead. As you can see in the logs mergerfs does receive the request which is almost certainly doesn't if you disable that cap. So the kernel is doing the proper checks and forwarding the request. It's just that mergerfs, running the request as uid 1000, doesn't have perms.
I meant that it's surprising for a regular user of mergerfs who wouldn't be familiar with the FUSE protocol. BTW, the reason I ended up using setcap is that it's recommended by restic: https://restic.readthedocs.io/en/latest/080_examples.html?highlight=root#full-backup-without-root I mention that only in case others end up running into the same problem.
It sounds like this is a linux limitation that we have to live with. Thanks for responding to the issue and thanks for all your hard work on this project! mergerfs has been very useful for me.
Ahh. I see why that would be useful for backup software.
Like I mentioned... I think if you just set the cap on mergerfs it should work as expected. The kernel will see that the client app has it, allow the request through, and mergerfs would then be allowed too. Assuming it isn't impacted by changing of effective uid... which I'm not positive about. Worth a try. Otherwise I can investigate adding an optional feature to do dynamic lookups and setting it on the fly. Can also talk with the FUSE folks and see if there is something else that could be done but AFAICT those are the only possibilities.
If it were possible to give mergerfs the capability and have it only be used if the caller also has the capability, that would be perfect. I tried it and it doesn't seem to work:
$ ps -ef | grep merger
root 904 1 0 09:03 ? 00:00:05 /run/wrappers/bin/mergerfs /mnt/disk* /mnt/storage -o rw,cache.files=partial,use_ino,dropcacheonclose=true,allow_other,minfreespace=50G,fsname=mergerfs,posix_acl=true
$ getcap /run/wrappers/bin/mergerfs
/run/wrappers/bin/mergerfs cap_dac_read_search,cap_setpcap=ep
The version of cat WITH the capability still can't read the file even though mergerfs also now has the capability. Also, mergerfs appears to still be getting calls from the version of cat WITHOUT the capability.
mergerfs-catnocap.strace.txt mergerfs-catwcap.strace.txt catnocap.strace.txt catwcap.strace.txt
The version of cat WITH the capability still can't read the file even though mergerfs also now has the capability. Also, mergerfs appears to still be getting calls from the version of cat WITHOUT the capability.
It's not. Look at the logs. With caps you see open request for the file. In the other you do not.
906 09:10:48.515879 openat(AT_FDCWD, "/mnt/disk2/bar", O_RDONLY|O_LARGEFILE) = -1 EACCES (Permission denied) <0.000013>
That doesn't exist in the catnocap log.
After looking into how caps work with changing user ids it explains why setting on the binary doesn't work.
If the effective user ID is changed from 0 to nonzero, then all capabilities are cleared from the effective set.
I can prevent the permitted capability set from being cleared with a particular syscall but the effective set is still cleared so I'd have to actively keep track of and set the caps each time I change credentials and pick them up from the incoming process. It'd be a bit of work to plumb that throughout the code. I'd also be a little concerned regarding the overhead though I really can't say what impact it would have till I added it and tested.
Honestly... it'd be easier if you just ran the process as root. If that's not something that works for you I can put it on the list of enhancements but can't guarantee when I'll get to it as most of my time spent on mergerfs recently is focused on v3 rewrite.
Yeah, I'm OK running the process as root (or perhaps managing the permissions via groups / posix ACLs).