Having one large, untracked file pegs single CPU core to ~100%

[savchenko]

Describe the bug Leaving stray ~700mb file in a repo results in gitui taking unreasonable amount of CPU time.

To Reproduce Steps to reproduce the behavior:

1. Add random (very) large file to the git root.
2. Open gitui

Expected behavior The file is ignored unless user explicitly adds it to the repository.

Screenshots N/A

Context (please complete the following information):

• OS/Distro + Version: Debian 13
• GitUI Version: gitui 0.27.0 2025-01-14 (99f6967)
• Rust version: rustc 1.86.0 (05f9846f8 2025-03-31)

Additional context Removing the file resolves the issue in its entirety.

[extrawurst]

Oh boy that sounds like an upstream issue with libgit2 and their slow status functions - @cruessler when can we migrate the status tab to gitoxide? 😉

[cruessler]

Well, from what I can tell, gitoxide’s status API already has everything we need (I’ve already experimented with it a bit in a small side-project of mine). It’s even being integrated into cargo package as we speak (https://github.com/rust-lang/cargo/pull/15534). To summarize, I’ve just moved it to the front of my oxidize to-do list. 😉

[cruessler]

@savchenko I merged #2673 earlier which switches the status tab to use gitoxide instead of libgit2. Would you be able to test the latest master to see whether your issue got fixed?

[savchenko]

@cruessler still happens unfortunately.

Steps to reproduce

1. cargo build --release from 6685f9adb3c887ce9e9b230ff28e82c9ec326d54
2. dd if=/dev/urandom of=random_big_file bs=1M count=1000
3. Launch freshly compiled gitui

The CPU usage is following more or less the sinusoidal curve:

https://github.com/user-attachments/assets/46dff383-c88f-4757-a349-7b6fe54cf471

The screen recording above is made from this repository with the 1Gb untracked file. I also see the spinner appearing in the top-left corner of the gitui UI.

[cruessler]

Thanks for the detailed steps!

I was able to reproduce following your instructions (also built from 6685f9a) and then selecting the file in the status view so that gitui started generating the diff.

This produced the following profile that I collected using samply record gitui: https://share.firefox.dev/4kWIQQh. The time is mostly spent in the following function in my case: https://github.com/gitui-org/gitui/blob/6685f9adb3c887ce9e9b230ff28e82c9ec326d54/asyncgit/src/sync/diff.rs#L204-L217

The first thing I’m going to try is to either convert get_diff to gitoxide or to add logic that avoids generating a diff in the first place for this use case.

[extrawurst]

The first thing I’m going to try is to either convert get_diff to gitoxide

That would be great.

or to add logic that avoids generating a diff in the first place for this use case.

not sure how you want to avoid generating the diff, as this is whats shown on the right for a file, right? or is this issue about non diff-able binary files?

[cruessler]

The first thing I’m going to try is to either convert get_diff to gitoxide

That would be great.

It would also be my preferred solution.

or to add logic that avoids generating a diff in the first place for this use case.

not sure how you want to avoid generating the diff, as this is whats shown on the right for a file, right? or is this issue about non diff-able binary files?

Yes, I was talking about the diff on the right. I would only want to try this if the first option doesn’t work. I was thinking about potentially not running the diff for large binary files or only running a somehow reduced version of the diff.

[savchenko]

Perhaps, as a temporarily workaround, maximum file size to display can be limited via options?

For example, if max_size_to_display is < than file size, then render "File size exceeded" in the "Diff:" section.

[cruessler]

@savchenko Sorry for the long wait! I was hoping the transition to gitoxide was more easy for that part, but it turns out there’s a few dependencies. Thanks for your suggestion, I’ll have a look at your proposal!