Status() is slow with a large number of untracked files

[tych0]

trafficstars

I have a tree with a large number of untracked files, and calling Status() is slow. When I profile it, it seems to be hashing all of the untracked files, which is unnecessary.

[rafi]

Even without untracked files, it takes me about ~12s for a repository with a large supposedly ignored ./node_modules/ directory. Deleting this directory reduced the Status() time to 1.5s, still slow, but much more faster. This means Status() doesn't take .gitignore into account when traversing working-directory.

[isacikgoz]

can confirm this issue, maybe there should be a way to take .gitignore files into account.

[dedelala]

Regardless of untracked files and ignores it is quite slow in large worktrees. I have been testing with https://github.com/mawww/kakoune On a clean shallow clone it's an order of magnitude slower than calling git status --porcelain After running a build that time is roughly doubled again.

[jfontan]

After a quick look there seems to be two main problems:

• .gitignore is applied as last step: https://github.com/src-d/go-git/blob/master/worktree_status.go#L140
• SHA-1 is calculated for all files: https://sourcegraph.com/github.com/src-d/go-git@791aea319719ee757cb862e83ec43b25113de2c1/-/blob/utils/merkletrie/filesystem/node.go#L117

Probably the biggest speedup could come from only calculating the SHA-1 of the file when needed. This may be when the file is in both trees and has the same size.

[andrewrynhard]

Is this planned for any upcoming release? It makes any tools we built based on go-git very painful to use.

[tcolgate]

This is really killing us too.

[dennis-tra]

I also ran into this issue - it's a pity, because this is such an amazing project, but this issue makes it for my use case almost impossible to use