Slow git status causes Turbo to hang

[samhh]

Verify canary release

• [X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

n/a

What package manager are you using / does the bug impact?

Yarn v2/v3/v4 (node_modules linker only)

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

2.0.6

Describe the Bug

I use Git-backed Jujutsu, so in my repo I very rarely run git status. Not doing so for a while can be laggy for reasons that are probably obvious to anyone familiar with Git's innards.

This causes Turbo caching to radically slowdown, so the extent that a task that usually completes and caches in a few seconds will hang up the system with 100% CPU. Notably this task outputs lots of small files to disk.

$ turbo run my-task &
$ turbo daemon -vvv
[...]
2024-07-04T16:19:15.650+0100 [DEBUG] turborepo_scm::package_deps: failed to use git to hash files: git error: 'git status' in /path/to/workspace exited with code unknown. Falling back to manual
<C-c>
$ git status
$ turbo run my-task # works

Expected Behavior

The task completes like usual, possibly with an additional slowdown comparable to running git status once first, which fixes the issue.

To Reproduce

Presumably it'd be possible to repro and fix if you can find a way to artificially slow Git down.

Additional context

This is a bit of an edge case given Git's ubiquity, so I wouldn't mind if you closed this as not fixing. I figured I'd at least report it for anyone else using Jujutsu and hitting this issue.

Related: #8122

[NicholasLYang]

Perhaps I'm not sufficiently familiar with git's innards, but why does not running git status for a while make it laggy? Does Jujutsu not use the fsmonitor daemon? And is that SCM log coming from the daemon or the turbo run process? If it's the former, could you try running with --no-daemon?

[samhh]

I believe there is fsmonitor support in Jujutsu via Watchman, which I currently have disabled. Presumably since git fsmonitor--daemon internally touches the same Git paths as git status the idea is it should sort of implicitly solve the lagginess for me?

The SCM log came from the Turbo daemon. Next time the lag builds up I'll try to remember to give --no-daemon a go.

[NullVoxPopuli]

git can also be slow on corporate machines with a number of security tools intercepting every disk i/o behavior

[anthonyshew]

Ultimately, this behavior is out of our hands. We depend on Git in a number of ways, so slowness in Git will mean slowness for Turborepo. Not depending on Git would require changes in the fundamental architecture of Turborepo, so I'll close this as a wont-fix.