fscrypt on CephFS does not recognize locked directories upon remount

[ossarchitect]

This issue is present in Ubuntu 23.10 with Kernel 6.6.7 from mainline. (6.6+ is required for fscrypt support on CephFS). fscrypt is installed via apt. What I am doing:

• sudo mount -t ceph [email protected]=/ /mnt
• cd /mnt/fscrypt setup
• mkdir cryptdir
• fscrypt encrypt cryptdir
• The result here is correct, I get "cryptdir is now encrypted, unlocked and ready to use." At this point I can fscrypt lock and unlock the directory When the directory is locked, the files inside are encrypted, with encrypted filenames. When it is unlocked. the files are decrypted.
• sudo umount /mnt
• sudo mount -t ceph [email protected]=/ /mnt
• fscrypt unlock cryptdir Result: [ERROR] fscrypt unlock: file or directory cryptdir is not encrypted The files and directories inside the directory are encrypted, but fscrypt does not recognize that they are. I tried locking the directory before umount and leaving it unlocked before umount. Neither works. I tried to find a solution online, but came up empty.

Am I missing something or is this a bug? And if it is the latter, do I need to file this with the kernel team for the CephFS driver, too?

[ebiggers]

This should work, and it does work on other filesystems. Please report this to the CephFS developers.

[ossarchitect]

Will do, thanks!

[ossarchitect]

I opened https://tracker.ceph.com/issues/63939 for this.

[ossarchitect]

In connection with the testing I frequently get the following error: $ fscrypt encrypt [ERROR] fscrypt encrypt: : readdirent : No buffer space available

This is particularly evident when trying to manipulate the formerly encrypted directory (lock/unlock/encrypt) that is marked as not encrypted after the remount.

[ossarchitect]

Interestingly the issue with 'is not encrypted' does not seem to be limited to umount and remount:

root@ubuntu:/mnt# mkdir test4 root@ubuntu:/mnt# fscrypt encrypt test4 Should we create a new protector? [y/N] Enter custom passphrase for protector "thecrypta": "test4" is now encrypted, unlocked, and ready for use. root@ubuntu:/mnt# vi test4/testfile root@ubuntu:/mnt# ls test4 testfile root@ubuntu:/mnt# fscrypt lock test4 "test4" is now locked. root@ubuntu:/mnt# ls test4 I1eTGrt5j2K08BlIdpUs++w4tFnJtes5JuY7n1,gja4 root@ubuntu:/mnt# fscrypt unlock test4 [ERROR] fscrypt unlock: file or directory "test4" is not encrypted root@ubuntu:/mnt#

For some reason the metadata read does not seem to work right. the error 'is not encrypted' is in metadata/policy.go in the function GetPolicy. It is triggered whem getPolicyIoctl gets the error unix.ENODATA So I infer the policy is not written properly on CephFS

[ossarchitect]

Update: the problem appears to be writing into the encrypted (unlocked) CephFS directory. Locking and unlocking of an encrypted empty directory works Locking of an encrypted directory with a file written in it does work (or at least does not produce an error) Unlocking of an encrypted directory with a file written in it does not. I get the error message "[ERROR] fscrypt unlock: file or directory "crypt" is not encrypted"

[ossarchitect]

Upgrade of the Ceph cluster to 18.2.1 (Reef) did not change the behavior.

[ebiggers]

Closing as this is now tracked in the CephFS bug tracker https://tracker.ceph.com/issues/63939.