multiple login nodes (multiple lustre client), how should I correctly apply the fscrypt tool to encrypt files under shared storage?

[voidspiral]

If I'm in an HPC environment with multiple login nodes, such as login1, login2..., login3..., HPC generally uses LDAP to manage account login information for these nodes. On login1, I log in with the user1 account and execute fscrypt setup, which creates a .fscrypt folder in the / directory. I then run fscrypt setup /mnt/lustre using login as the login key, followed by fscrypt encryption dir and locking the file. However, when I log in to login2 with the user1 account and execute fscrypt unlock, the fscrypt program cannot find the metadata information located in the / directory, and it prompts the following error.

"ttt" is encrypted with fscrypt.

Policy:   2700eb1e6f935208b14ca12a4aff3b25
Options:  padding:32  contents:AES_256_XTS  filenames:AES_256_CTS  policy_version:2
Unlocked: No

Protected with 2 protectors:
PROTECTOR         LINKED  DESCRIPTION
                          [cannot follow filesystem link "UUID=1cddd903-cbe0-4491-bf06-a98504e8b27c\nPATH=/\n": protector metadata for a50d4adfab0ce6a2 not found on filesystem /]
a405cfc9c27eebc7  No      custom protector "Recovery passphrase for ttt"

In this scenario with multiple login nodes, how should I correctly apply the fscrypt tool to encrypt files under shared storage?

[ebiggers]

fscrypt login protectors are always stored in the root directory, for the reasons mentioned at https://github.com/google/fscrypt/issues/164#issuecomment-554024149. So they're local to the system.

I think this is the first time that the use case of multiple systems with a shared login and shared encrypted directory has been brought up. Perhaps because the filesystems on which upstream Linux supports native encryption have traditionally all been local filesystems. The first non-local one, CephFS, was added in Linux v6.6 which was just released a few days ago. The Lustre support has been around a bit longer but isn't upstream.

It might be helpful to check with the Lustre developers if they have something in mind for your use case already. But, I think the best advice I can give for now, for this use case, is to just use a custom passphrase protector instead of a login protector.

It would be possible to add support to fscrypt for per-filesystem login protectors, which would support this use case. It's worth considering, but would it add complexity and would come with the disadvantages mentioned at https://github.com/google/fscrypt/issues/164#issuecomment-554024149 so it probably shouldn't be the default option. I'd first like to get a good sense that it would be useful to a significant number of users.

I guess another possibility would be just an option in /etc/fscrypt.conf that would override the mountpoint where login protectors are stored. That would handle the special case where all login protectors are stored / should be stored on one mountpoint that is not the root filesystem.