Hold down arrow key to switch file, diff window stop render.

[LucaXiang]

Describe the bug

1. Unstaged Changes stop update
2. Exit with error message "cannot kill child process"

To Reproduce

1. Enter Files Window
2. Hold Down the arrow keys (up or down) to switch files, (Single click do not happen)
3. Un staged changes or diff window stop update

Expected behavior Un staged Changes windows should to update.

Screenshots:

Desktop:

• OS: Windows
• Lazy Git Version : v0.27.4

Additional context I tried it in WSL2 (Ubuntu) and it doesn't have this problem

v0.35 still happen

[LucaXiang]

I tried it in v25 v26 v27 have same problem

[MikeVensel]

I am still having this issue very frequently on v0.31.4

[LucaXiang]

I am still having this issue very frequently on v0.31.4

guy i recommend you use wsl2 (windows subsystem linux).

[MikeVensel]

I can understand suggesting WSL 2 as a work around temporarily, but I don't think it is unreasonable to ask for a fix for this issue. Lazygit claims to support Windows and this bug makes using the tool on Windows irritating.

[Aziks0]

This is a major issue for Windows users, unfortunately. I got to a point where I couldn't use lazygit for more than a few seconds after running it (unless I wasn't using the file tree, obviously). I even stopped using it for the last 2 months.

---

I've done some digging by checking the log tho, here's what I've found:

1. After executing lazygit at the source of my repo, the selected file is the folder .tasks. The main view appears and reports error: Could not access '.tasks/null', which is expected.

• A first RunCommand command="git diff --submodule --no-ext-diff --unified=3 --color=always --no-index -- /dev/null .tasks" is executed, it seems to work without error.
• A second RunCommand command="git diff [...] .tasks" is executed, and is followed by the error error when running cmd task: invalid argument and then (I guess) the "goroutine" exits itself.

2. I select the next file in the tree, .tasks/bump.v. The main view appears instantly, it contains the file diff, nothing weird here.

• A first RunCommand command="git diff --submodule --no-ext-diff --unified=3 --color=always --no-index -- /dev/null .tasks/bump.v" is executed, followed by the error error when running cmd task: invalid argument. It is not followed by error status 1.
• A few seconds after the first one, a second RunCommand command="git diff [...] .tasks/bump.v" is executed, followed by the error error when running cmd task: invalid argument. It is followed by the error error when running cmd task: TerminateProcess: Access denied, and then error status 1.

3. I hold down ↓ (down arrow key) to walk down the file tree. The main view freezes.

• Tons of RunCommand command="git diff [...] {FILE_NAME}", but not a single invalid argument, nor TerminateProcess: Access denied, nor exit status 1.

4. I exit lazygit, and, surprise surprise: cannot kill child process.

---

Ok, so now, what happens if I do everything again, but instead of holding down ↓, I press it multiple times ?

1. Executing lazygit at the source of my repo.

• same as before

2. Pressing ↓ multiple times.

(I spare you the very-very-very-long screenshot, but the log file is available at the end of my comment)

• RunCommand command="git diff [...] {FILE_NAME}"
• Error invalid argument
• Error TerminateProcess: Access Denied
• exit status 1
• Repeat

The main view doesn't freeze, it shows what is expected: the files diff.

3. I exit lazygit, and, surprise surprise: no issue.

Holding <↓> log file Pressing <↓> multiple times log file

[nikit-psl]

A similar thing is happening for me when I'm using lazygit inside toggleterm in neovim. Terminal - Alacritty OS - Windows 11

It's working fine when I'm using lazygit directly in Alacritty though. Also, this only happens when I keep the down key pressed and doesn't happen when I keep the up key pressed.

[Aziks0]

I've done some debugging, but I'm stuck right now.

What happens when the view freezes

It's a bit difficult to explain because of goroutines, but I'll try my best.

1. NewTask

gui.newCmdTask() is called with the command git diff [...] {FILE_NAME} to find the diff of the selected file. The gui.newCmdTask() function has this line, it's where the beginning of the issue stands: https://github.com/jesseduffield/lazygit/blob/e68093fe9974296fe5afb22ec8e0adb1eb2f4316/pkg/gui/tasks_adapter.go#L38

manager.NewTask() is called, the function starts a goroutine — NewTask.func1, and then returns nil.

2. NewTask.func1

NewTask.func1 has a mutex that is locked at the start of the function, and is unlocked at the end (with a defer statement): https://github.com/jesseduffield/lazygit/blob/3bf0c9ef441f9343172ed71b5e5947c69e66ac9a/pkg/tasks/tasks.go#L258-L259

After the lock, there are 2 important things, the first one looks like this : https://github.com/jesseduffield/lazygit/blob/3bf0c9ef441f9343172ed71b5e5947c69e66ac9a/pkg/tasks/tasks.go#L265-L278

And the other one is this goroutine (NewTask.func1.1): https://github.com/jesseduffield/lazygit/blob/3bf0c9ef441f9343172ed71b5e5947c69e66ac9a/pkg/tasks/tasks.go#L280-L286

f(stop) calls manager.NewCmdTask(). When it returns, the channel notifyStopped is closed.

When the bug happens, self.stopCurrentTask() is called, so those 2 lines are executed: https://github.com/jesseduffield/lazygit/blob/3bf0c9ef441f9343172ed71b5e5947c69e66ac9a/pkg/tasks/tasks.go#L274-L275

• the channel stop is closed
• NewTask.func1 waits for notifyStopped to be closed

Something that needs to be noted: self.waitingMutex is NOT UNLOCKED until notifyStopped is closed.

3. NewCmdTask

NewCmdTask does many things, so I'm going to make a "summary function":

select {
case <-stop:
    return nil
default:
}

done := make(chan struct{})

scanner := bufio.NewScanner(r)
scanner.Split(bufio.ScanLines)

// `NewCmdTask.func1`
go utils.Safe(func() {
outer:
    for {
        // Some lines
        ok := scanner.Scan()
        // Some other lines
        if !ok {
            self.onEndOfInput()
            break outer
        }

    self.refreshView()
    close(done)
}) // End of `NewCmdTask.func1`

<-done
return nil

• if stop is closed, the function returns instantly
• a channel done is created
• a scanner is created (it scans stdout of the command git diff [...] {FILE_NAME})
• the function waits for done to be closed before returning

In parallel (NewCmdTask.func1):

• a line is scanned
• if there are no more lines to scan, the for statement breaks out
  • the view is refreshed
  • done is closed

And this is where it all comes together, when there are no more lines to read:

• the view is refreshed with the content of git diff [...] {FILE_NAME}
• done is closed
• NewCmdTask returns
• notifyStopped is closed
• self.waitingMutex is unlocked (because NewTask.func1 returns)

BUT, we are talking about a bug, so something messes up at some point, right ?

Well, I don't know why, but it seems like the scanner doesn't scan anymore...

This line hangs (?) indefinitely: https://github.com/jesseduffield/lazygit/blob/3bf0c9ef441f9343172ed71b5e5947c69e66ac9a/pkg/tasks/tasks.go#L172

Stack trace of NewCmdTask.func1

??? (?:-1)
runtime.systemstack_switch (...\go\src\runtime\asm_amd64.s:436)
runtime.cgocall (...\go\src\runtime\cgocall.go:167)
syscall.SyscallN (...\go\src\runtime\syscall_windows.go:538)
syscall.Syscall6 (.:0)
syscall.ReadFile (...\go\src\syscall\zsyscall_windows.go:1024)
syscall.Read (...\go\src\syscall\syscall_windows.go:380)
internal/poll.(*FD).Read (...\go\src\internal\poll\fd_windows.go:427)
os.(*File).read (...\go\src\os\file_posix.go:31)
os.(*File).Read (...\go\src\os\file.go:119)
bufio.(*Scanner).Scan (...\go\src\bufio\scan.go:215)
github.com/jesseduffield/lazygit/pkg/tasks.(*ViewBufferManager).NewCmdTask.func1.4 (...\lazygit\pkg\tasks\tasks.go:172)
github.com/jesseduffield/lazygit/pkg/utils.Safe.func1 (...\lazygit\pkg\utils\utils.go:104)
github.com/jesseduffield/lazygit/pkg/utils.SafeWithError (...\lazygit\pkg\utils\utils.go:115)
github.com/jesseduffield/lazygit/pkg/utils.Safe (...\lazygit\pkg\utils\utils.go:104)
github.com/jesseduffield/lazygit/pkg/tasks.(*ViewBufferManager).NewCmdTask.func1.7 (...\lazygit\pkg\tasks\tasks.go:158)
runtime.goexit (...\go\src\runtime\asm_amd64.s:1571)

Because of this:

• view is not refreshed
• done is not closed
• NewCmdTask does not return
• notifyStopped is not closed
• self.waitingMutex is not unlocked
• ⇒ every time NewTask is called, it hangs on self.waitingMutex.Lock()
• ⇒ The command git diff [...] {FILE_NAME} is never called anymore
• ⇒ The view never refreshes

I don't know what happens with the scanner. I don't understand what happens with the scanner.

Pls help @jesseduffield. I'm going crazy.

[jesseduffield]

great investigation @Aziks0. Some things I've noted while taking a look at the code just now:

1. The start of the process is typically to call gui.newPtyTask, which will itself only call gui.newCmdTask if there's no custom pager being used OR if we're on windows because that gui.newPtyTask function is called is overridden for windows in pkg/gui/pty_windows.go (this is because the pty library doesn't yet support windows (see https://github.com/creack/pty/pull/109). I wonder if the problem is that the git command does use a pager (e.g. less) and so never returns an EOF to the scanner, meaning we get stuck on scanner.Scan(). Scanners block while waiting for new content, but we could swap out the scanner with a plain old reader and would then be able to deal with a lack of new content.

2. The in-built process killing method doesn't handle child processes of a windows process. I've got a PR to handle this, adapted from the approach we take in lazydocker. @Aziks0 would you be able to see if the issue persists on this PR? https://github.com/jesseduffield/lazygit/pull/2006

[Aziks0]

@jesseduffield

1. I've tried with and without a pager a few months ago, it didn't change anything to not have one. I've retried today, no difference.
2. The issue persist, unfortunately :(. It is not better or worst, just the same as before.

[Aziks0]

Btw, if I hold down ↓ instantly after executing lazygit, the issue appears instantly. If I wait a few seconds before holding down ↓/↑, the issue appears more randomly.

https://user-images.githubusercontent.com/36425380/174432026-88c1d836-33be-42fe-978c-782f2ea7af49.mp4

[adriancostin6]

I am also experiencing this annoying issue. Really liked using lazygit, but no I'm back to using the command line git tools, as it is better than pulling my hair out every time the diff view freezes. Any updates on this? Can I do something to help?

[mark2185]

Could you maybe try the latest master? There were a few updates regarding gocui

[LucaXiang]

After upgrading to the latest lazygit, the problem never occurred again

[AndrewSav]

may be we can make a release soon then? it's a pretty major thing to push out IMO!

[mark2185]

Last release was a month ago, does it still happen with v0.37.0?

[AndrewSav]

Yes, still happening on v0.37.0

Unstaged changes stop updating too.

[mark2185]

@AndrewSav and just to confirm, you don't have any issues on latest master as well?

[AndrewSav]

@mark2185 I can try latest master if there is a release that is built from the latest master.

[mark2185]

@mark2185 I can try latest master if there is a release that is built from the latest master.

There isn't, I meant could you try building the latest master and see if it happens there as well?

[adriancostin6]

@mark2185 done. results below. os: windows 10

[adriancostin6]

hello, just wanted to follow up to see if anyone is still looking at this?

[AndrewSav]

Hey, just wanted to share my experience. I did a few things which did not completely eliminate the occurrence of the issue, but brought down the frequency with which it's occurring. I think this may be a combination of factors:

• It happen less with (new) Windows Terminal. If using alacrity or conemu I can reproduce it much faster
• It happens less if git is NOT installed from scoop, scoop adds a shim which adds friction, again when git is installed from scoop I can reproduce it faster
• When lazygit gets to a bugged state there are one or more suspended git processes sitting in the task manager. I believe this explains scanner not scanning - the pipe is there but the process is suspended. If I run Get-Process git | Stop-Process the panels start refreshing again. I do not offer this as a work around, since it's much simpler just to quit lazygit and re-run it, just as an additional data point.

Replacing scanner with a reader as suggested above might help, and also if someone smart enough can figure out why we get these suspended git processes, we might finally resolve this issue.

Update

Git for Windows also has a shim: C:\Program Files\Git\cmd has a shim and C:\Program Files\Git\mingw64\bin has the actual binary. Once I put the latter at the top of the path like this:

I can no longer reproduce the issue. (Make sure that your lazygit process has the environment block that reflects this change, rebooting will take care of this, but there are easier but less reliable ways)

[AndrewSav]

I attempted to do a minimal fix so that when the scanner is blocked it does not hang the task. It seems to work for me even with all the shims and conemu. Can someone please test? (Not a go programmer, please be gentle)

• https://github.com/AndrewSav/lazygit/tree/fix-1265 (this is branched at tag v0.38.2, since master had some new unrelated issues that interfered with my testing)
• diff

I built an exe for testing and uploaded it here

[AndrewSav]

So the above seems to work for me in terms of fixing the display issue, however we still have suspended git processes left behind. I think this is a timing issue, and what might be happening is this:

• (1) lazygit spawns git process, shim is on path
• (2) shim is started and is trying to create child process
• (3) lazygit detects that current item in the UI changed and tries to start a task of running git command to get the diff window
• (4) lazygit attempts to kill the old process (1) and the children
• (5) first, lazygit enumerates all the children of the shim, there are none so far
• (6) at this point shim creates child process suspended and connects the stdout pipe
• (7) now lazygit kills the shim, but does not kill the child because it was not there when it enumerated processes (5) for children
• (8) since the shim did not have a chance to un-suspend the child before getting killed, the suspended child persists

At this point we are left with a suspended process which both causes the scanner to block and remains there until someone deletes it manually or the system is rebooted, we just leaked this process.

I feel that fixing this may be quite tricky especially if several shims are involved: the scoop one and the git for windows own one. The workaround here for a user, would be not to use shims and run the actual git process directly.

[adriancostin6]

@AndrewSav, thanks for all the work you put in for this seemingly dead issue, that has been here for quite a while.

You just opened my eyes to why this happens with your clear explanations. Kudos to you!

I do use scoop, and never thought of the shims. Will try installing them directly, and see if I get better results.

[jesseduffield]

Great investigation @AndrewSav . I've googled around a bit to see if there's some way to resolve this but haven't had any luck so far

[jesseduffield]

Actually this might help: https://gist.github.com/hallazzang/76f3970bfc949831808bbebc8ca15209

[AndrewSav]

@jesseduffield, thanks, any point submitting PR with the display issue fix (above)?

[AndrewSav]

@jesseduffield I still would like to know if you would be interested in the PR. I used the patched version this week and in my workflow I have not encountered any issues

[AndrewSav]

I'm using get-process git | ?{($_.Threads.ThreadState -eq "Wait").Count -eq $_.Threads.Count } to kill all those dozens of git.exe left behind in the meanwhile.