v4: Mounting VFS export fails with "No such file or directory"

[tobiaslangner]

I run a ganesha-nfs docker container (image from here with tagv6.0.7-stable-6.0-pacific-centos-stream8 - yeah I know, it's not the newest...) with the primary goal of serving a CEPH filesystem through NFS. This works fine and very reliably.

Now I wanted to dual-use the NFS container to also serve a couple of local directories from my host by bind-mounting those directories into the ganesha-nfs container and then exposing them with VFS FSAL. My /etc/ganesha/ganesha.conf looks as follows (with ceph-specific and unrelated things left out):

EXPORT {
    FSAL {
        name = "CEPH";
        filesystem = "dfs";
    }

    export_id=100;
    #
    # Path into the cephfs tree.
    #
    # Note that FSAL_CEPH does not support subtree checking, so there is
    # no way to validate that a filehandle presented by a client is
    # reachable via an exported subtree.
    #
    # For that reason, we just export "/" here.
    path = "/";
    pseudo = "/dfs";
    access_type = "RW";
    squash = "none";
    attr_expiration_time = 0;
    protocols = 4;
    transports = "TCP";
}

EXPORT
{
    FSAL
    {
        Name = VFS;
    }

    Export_Id = 102;
    Path = /photos;
    Pseudo = /photos;
    Protocols = 4;
    Transports = "TCP";
    Squash = "None";
}

Mounting the ceph export through mount -t nfs4 host:/dfs <target> works without problems. Attempting to mount the photos share through mount -t nfs4 host:/photos <target> fails with mount.nfs4: Operation not permitted. When mounting, ganesha's debug log shows the following lines which seem relevant to the error:

ganesha.nfsd-58[svc_9] nfs4_op_lookup :NFS4 :DEBUG :name=photos
ganesha.nfsd-58[svc_9] release :FSAL :DEBUG :Releasing live hdl=0x561adec7b8f0, name=photos, don't deconstruct it
ganesha.nfsd-58[svc_9] nfs4_op_lookup :EXPORT :DEBUG :PSEUDO FS JUNCTION TRAVERSAL: Crossed to /photos, id=102 for name=photos
ganesha.nfsd-58[svc_9] complete_op :NFS4 :DEBUG :Status of OP_LOOKUP in position 2 = NFS4_OK, op response size is 4 total response size is 92
ganesha.nfsd-58[svc_9] process_one_op :NFS4 :DEBUG :Request 3: opcode 10 is OP_GETFH
ganesha.nfsd-58[svc_9] fsal_common_is_referral :FSAL :DEBUG :Checking attrs for referral, handle: 0x561adec8cc50, valid_mask: 0, request_mask: 82, supported: 0
ganesha.nfsd-58[svc_9] vfs_open_by_handle :FSAL :DEBUG :Failed with Operation not permitted openflags 0x00000000
ganesha.nfsd-58[svc_9] find_fd :FSAL :DEBUG :Failed with Operation not permitted openflags 0x00000020
ganesha.nfsd-58[svc_9] fsal_common_is_referral :FSAL :EVENT :Failed to get attrs for referral, handle: 0x561adec8cc50, valid_mask: 0, request_mask: 82, supported: 0, error: Forbidden action

Any ideas what is going on here or what I could investigate to get to the bottom of this? Thanks heaps!

[tobiaslangner]

The research for the post pointed me to this thread which pointed out that the problem could be ganesha requiring privileged access for this to work. And alas, running the container as privileged indeed makes the mount command succeed. Indeed, it is sufficient to grant it the capability SETFCAP (see this for more).

Is this working as intended?

[dang]

It's working as intended, yes. Ganesha needs the open_by_handle_at() to be able to turn a NFS object reference into a POSIX file handle so that we can do I/O on it. The kernel developers have determined that using this call requires elevated permissions, so Ganesha needs elevated permissions to serve local files. It's a fundamental incompatibility between NFS and POSIX.

[ffilz]

Yes, this is working as expected.

[ffilz]

I just added some text to the wiki documentation for FSAL_VFS.

[xiaods]

please closed by resolved