duc doesn't mention restricted folders on NFS mount

[maressyl]

Hi,

While indexing a large NFS share (~ 1 Po), I noticed a couple hundreds of To were missing from my grand total. Despite using -v, duc never complained about any folder I would not have access to, but I suspect the missing To are in such undetermined places.

Running a few tests, I noticed ls has a slightly different behavior in local and on the NFS share.

Locally :

$ ls -l test/
ls: impossible d'ouvrir le répertoire 'test/': Permission non accordée

On the NFS share :

$ ls -l test/
ls: lecture du répertoire 'test/': Permission non accordée
total 0

In both cases the erreor message is the same and echo $? display a code 2, but it appears as if the NFS share returned anyway an empty file list.

• Is it possible that duc is tricked by this behavior ?
• Has anyone experienced such a behavior on a NFS4 share ?
• Is there any known workaround ?

Some details regarding the mount (slightly redacted using ...) :

mount -l
...:/ on /... type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5,clientaddr=...,local_lock=none,addr=...)

Thanks a lot for your help, and this very useful piece of software !

Best, Sylvain

[l8gravely]

"Sylvain" == Sylvain Mareschal @.***> writes:

Ooops, sent the previous reply before I meant to.

Sylvain> While indexing a large NFS share (~ 1 Po), I noticed a couple Sylvain> hundreds of To were missing from my grand total. Despite Sylvain> using -v, duc never complained about any folder I would not Sylvain> have access to, but I suspect the missing To are in such Sylvain> undetermined places.

I'm impressed, are you indexing this all in one go? How long does it take and how large is the index you generate? Can you please grab the latest HEAD code from https://github.com/zevv/duc and compile it and make sure to do an index of a problematic directory using the --debug switch please?

Also, I haven't run an NFSv4 setup at all, so I'll have to try and setup something and run some tests. Are you using ACLs at all?

Sylvain> Running a few tests, I noticed ls has a slightly different Sylvain> behavior in local and on the NFS share.

duc doesn't use ls, so these tests aren't really relevant. But since I notive you're using a non-english LANG, that might also be causing problems. I haven't really tested much beyond UTF-8 as my default encoding now.

Sylvain> Locally :

Sylvain> $ ls -l test/ Sylvain> ls: impossible d'ouvrir le répertoire 'test/': Permission non accordée

Sylvain> On the NFS share :

Sylvain> $ ls -l test/ Sylvain> ls: lecture du répertoire 'test/': Permission non accordée Sylvain> total 0

Sylvain> In both cases the erreor message is the same and echo $? Sylvain> display a code 2, but it appears as if the NFS share returned Sylvain> anyway an empty file list.

Sylvain> * Is it possible that duc is tricked by this behavior ?

Maybe. Can you give us more details on your setup?

Sylvain> * Has anyone experienced such a behavior on a NFS4 share ?

I haven't used NFSv4 yet, so I suspect you're a trail blazer here.

Sylvain> * Is there any known workaround ?

Sylvain> Some details regarding the mount (slightly redacted using ...) :

Sylvain> mount -l Sylvain> ...:/ on /... type nfs4 (rw,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5,clientaddr=...,local_lock=none,addr=...)

Sylvain> Thanks a lot for your help, and this very useful piece of software !

Thank you for the kind words, though I'm not the main programmer here, I'm just the guy who trys to maintain and continue releases when I can. I'm not so good as it. LOL!

Can you possibly share with us a same DB showing the problem? Or the --debug output from a duc index run?

Thanks, John