Symlinks from wine result in walker infinite loop

[harrypotterIUP]

Checklist

• [X] I have read through the manual page (man fzf)
• [X] I have searched through the existing issues
• [X] For bug reports, I have checked if the bug is reproducible in the latest version of fzf

Output of fzf --version

0.55.0 (fc69308)

OS

• [X] Linux
• [ ] macOS
• [ ] Windows
• [ ] Etc.

Shell

• [X] bash
• [ ] zsh
• [ ] fish

Problem / Steps to reproduce

A simple call fzf in my home folder takes up all CPU and memory resources and seems to run forever. Initially, I assumed that my laptop is just too slow for fzf (see #4014), but now it looks more like a problem with an infinite loop of the fzf walker.

I noticed that the issue disappears with fzf --walker=file,hidden (omitting follow). A subsequent investigation of all relevant subfolders which contain symlinks (with find . -type l | sed 's/.\/\([^/]\+\).*/\1/p' | uniq) showed a lot of wine prefixes.

This suspicion indeed turned out to be the issue: The problem disappears with fzf --walker-skip=drive_c,dosdevices (two automatically created folders/symlinks in any wine prefix) and the walker finishes very quickly.

As expected, the issue also disappears when disabling the internal walker and using FZF_DEFAULT_COMMAND='find .' fzf instead. When making find follow symlinks find -L . the command runs very long and there are lots of "Too many levels of symbolic links" and "File system loop detected" errors.

I am a little surprised, though, that I should be the first one to report this issue, since wine is a widespread program? And the symlinks from wine won't be the only case that lead to infinite loops I guess? Interested to hear if others can reproduce the issue and how to deal with symlinks in general. :+1:

[LangLangBart]

Interested to hear if others can reproduce the issue and how to deal with symlinks in general.

I reproduced the issue with fnm[^1] where enabling its shell setup in my zshrc causes a new symlink directory to be created for every shell instance, quickly resulting in thousands of symlinks under ~/.local/state/fnm_multishells.

The fnm's maintainer provided more details here:

• https://github.com/Schniz/fnm/issues/1157#issuecomment-2143788670

Tools like fzf, rg, and fd take significantly longer when following symlinks in these directories.

cd ~/.local/state/fnm_multishells
find . -type l -depth 1 | wc -l
    4608

# fzf version: 0.55.0
unset FZF_DEFAULT_COMMAND
time command fzf --walker=file,follow --filter ''
user=86.46s system=241.46s cpu=232% total=2:21.14

# fd version: 10.2.0
time command fd --follow --color=never
user=163.76s system=582.79s cpu=496% total=2:30.25

# ripgrep version: 14.1.0
time command rg --follow --files --color=never
user=298.48s system=1245.37s cpu=614% total=4:11.40

Hence, I would suggest this isn't a bug but a design choice by fzf to enable follow by default. Users should be aware and use options like fzf's --walker-skip= to avoid unnecessary directories.

[^1]: GitHub - Schniz/fnm: 🚀 Fast and simple Node.js version manager, built in Rust

[harrypotterIUP]

Thank you very much for looking into this!

Hence, I would suggest this isn't a bug but a design choice by fzf to enable follow by default

Yes, I agree. In previous versions, find -L was also used. So whoever has issues with symlinks with the walker now, already had them before with find -L, too. No need to change the default follow.

Nevertheless, I think there are still two points worth addressing:

1. Is it possible to make the walker smarter to recognize symlink infinite loops? In other words, there would be no need to deactivate follow if there's a way to avoid that heavy overload caused by looping/repeatedly following the same symlinks.
2. Is it possible to make --walker-skip smarter? For example, one of the symlinks created by wine is wine/drive_c/users/<username>/Downloads. Setting --walker-skip=drive_c is excluding the whole wine file structure (excluding program files, too) which very well might contain files I want fzf to match, yet setting --walker-skip=Downloads might be even worse if my actual downloads folder shares the same name. So maybe a feature like --walker-skip=drive_c/users or even --walker-skip=drive_c/users/*/Downloads for a more precise directory name blacklist could help?

[junegunn]

Thanks for your interest in the project.

1. Is it possible to make the walker smarter to recognize symlink infinite loops?

If I'm not mistaken, the current walker implementation does detect loops.

See https://github.com/charlievieth/fastwalk/blob/17800a2b07b0829938668b367132fff7cdd7e0e7/entry_filter_unix.go#L21-L22

$ cd /tmp

$ mkdir -p foo/bar

$ ln -s ../../foo foo/bar/foo

$ find -L foo
foo
foo/bar
foo/bar/foo

$ FZF_DEFAULT_COMMAND= fzf --walker dir,file,follow --walker-root /tmp/foo --filter ''
/tmp/foo
/tmp/foo/bar
/tmp/foo/bar/foo

2. Is it possible to make --walker-skip smarter?

fzf focuses on being a good interactive filter rather than a versatile file system walker. The built-in walker can be a good default for many users, but it lacks options that a proper walker program should have, such as .gitignore support. If you want a better walker, you're better off finding a dedicated walker program like fd and using it as your FZF_DEFAULT_COMMAND. fzf will not compete in that department with those programs, because, Unix philosophy.

See

• https://github.com/junegunn/fzf?tab=readme-ov-file#respecting-gitignore
• https://github.com/junegunn/fzf?tab=readme-ov-file#settings

[junegunn]

2. So maybe a feature like --walker-skip=drive_c/users or even --walker-skip=drive_c/users/*/Downloads for a more precise directory name blacklist could help?

See #4107. It wasn't difficult to add support for multi-component ignore patterns. But no wildcard support.

[RedSnt]

For me the fix was just adding --walker-skip z: as z: is the symlink pointing to root.
Even better, if custom symlinks are added to some wineprefixes (I have plenty via bottles and heroic games launcher and steam), is using --walker-skip dosdevices - that skips all wineprefix "drives"/symlinks.