recursion limit exceeded running on smb mounts

[pgwipeout]

When a directory is mounted over smb via samba on linux, running rdfind inside that directory spams:

recursion limit exceeded
recursion limit exceeded
recursion limit exceeded
recursion limit exceeded
recursion limit exceeded

and never completes.

Observed the following errors in the kernel log:

[ 1588.565758] CIFS VFS: Close unmatched open
[ 1588.566208] CIFS VFS: Close unmatched open
[ 1588.566521] CIFS VFS: Close unmatched open
[ 1588.566856] CIFS VFS: Close unmatched open
[ 1588.567192] CIFS VFS: Close unmatched open
[ 1588.567573] CIFS VFS: Close unmatched open
[ 1588.567809] CIFS VFS: Close unmatched open
[ 1588.568044] CIFS VFS: Close unmatched open
[ 1588.568425] CIFS VFS: Close unmatched open
[ 1588.568691] CIFS VFS: Close unmatched open

When a directory is mounted in wsl1 over smb via samba, it also includes errors such as:

Dirlist.cc::handlepossiblefile: This should never happen. FIXME! details on the next row:
possiblefile="./master/.hardinfo"
Dirlist.cc::handlepossiblefile: This should never happen. FIXME! details on the next row:
possiblefile="./master/build/build/ouya/menu/initrd/data"

along with the recursion limit exceeded errors.

Samba host is ubuntu 20.04, running kernel 5.6.11. Linux client is ubuntu 19.10. WSL client is ubuntu 18.04. "This is rdfind version 1.4.1" Issues do not manifest when running over nfs.

[pgwipeout]

Update: The Dirlist.cc::handlepossiblefile: This should never happen. FIXME! details on the next row: errors occur with NFS in async mode.

[dstromberg]

I'm getting "recursion limit exceeded" as well, but it's on a local XFS filesystem - no CIFS or NFS involved.

The directory hierarchy it's scanning is large, but shouldn't be that deep. I'm computing an average and maximum depth right now, but as I said, it's a large hierarchy so it'll probably take a few days.

[dstromberg]

BTW, I got the error in a dry run.

[dstromberg]

Here are the statistics for my XFS. The numbers summarized are just the count of /'s in each file in the filesystem. 239 (the maximum) is deeper than I expected, but it still shouldn't cause a stack depth problem: minimum 0 maximum 239 range 239 count 284893128 sum 2592676051 arithmetic_mean 9.100521550663728 geometric_mean 0.0 median 9.0 mode 8 standard_deviation 2.334301 median_absolute_deviation 1.000000 mean_absolute_deviation 1.772618

[dstromberg]

...that is, unless there are some large locals in many stack frames.

[dstromberg]

I increased maxdepth with: --- Dirlist.cc.t 2020-05-30 08:36:08.621031666 -0700 +++ Dirlist.cc 2020-05-30 08:36:11.109063875 -0700 @@ -22,7 +22,7 @@ #include "Dirlist.hh" #include "RdfindDebug.hh" //debug macros

-static const int maxdepth = 50; +static const int maxdepth = 500;

int Dirlist::walk(const std::string& dir, const int recursionlevel)

...but then my rdfind process got killed - presumably by the OOM killer.

So here's a vote for that database suggestion.

[pauldreik]

Hmm, this does indeed need better handling.

[kwhitefoot]

I have a similar problem on a USB connected Toshiba Canvio 4TB disk that I use for backup. The disk is about 90% full according to df.

rdfind -makehardlinks true -sleep 100ms backups Now scanning "backups"recursion limit exceeded recursion limit exceeded Terminate called after throwing an instance of std::bad_alloc what(): std::bad_alloc Aborted

The machine is an Acer Aspire One 725 with 4 GB ram, running Linux Mint 18.3 Sylvia, kernel 4.8.0-53-generic,

Is there a work-around?

Edit: added command line

[dstromberg]

On Fri, Jul 2, 2021 at 6:34 AM kwhitefoot @.***> wrote:

I have a similar problem on a USB connected Toshiba Canvio 4TB disk that I use for backup. The disk is about 90% full according to df.

recursion limit exceeded Terminate called after throwing an instance of 'std::bad_alloc what(): std::bad_alloc Aborted

The machine is an Acer Aspire One 725 with 4 GB ram, running Linux Mint 18.3 Sylvia, kernel 4.8.0-53-generic,

Is there a work-around?

I think this is because junctions (sometimes?) look like hard linked directories, and can create a circuit. Without a device#+inode#, it's really hard to avoid retrraversing.

I reported a related bug here: https://github.com/PowerShell/Win32-OpenSSH/issues/1661

[kwhitefoot]

junctions (sometimes?) look like hard linked directories, and can create a circuit. Without a device#+inode#, it's really hard to avoid retrraversing. I reported a related bug here: PowerShell/Win32-OpenSSH#1661

Looks like I forgot to include an important piece of information: the disk in questin is formatted as ext4 not NTFS.

Perhaps it's related but junctions are not involved here as hardlinked directories are (as far as I know) not used in Ext4 disk formats. At least userland tools refuse to create hard links to directories.