In Isospin (Linux distribution), "gitstatusd failed to initialize": hangs on mkfifo

[bradfeehan]

Thanks for a great shell setup with powerlevel10k and gitstatusd 👏

I've used it successfully on my Mac for ages, but starting to use my dotfiles on Linux machines, my shell hangs indefinitely sometimes when opening a new shell. If I press Ctrl+C to interrupt it, I get this error:

  [ERROR]: gitstatus failed to initialize.


  Zsh log (/tmp/spin/gitstatus.POWERLEVEL9K.1000.110281.1712129070.1.xtrace.log):

    +(anon):7> setopt monitor
    +(anon):9> ((  ! _GITSTATUS_STATE_POWERLEVEL9K  ))
    +(anon):10> [[ -r /proc/version && 'Linux version 6.5.0-1016-gcp (buildd@lcy02-amd64-061) (x86_64-linux-gnu-gcc-12 (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #16~22.04.1-Ubuntu SMP Sat Mar  9 00:58:37 UTC 2024' == *Microsoft* ]]
    +(anon):13> print -rn
    +(anon):14> zsystem flock -f lock_fd /tmp/spin/gitstatus.POWERLEVEL9K.1000.110281.1712129070.1.lock
    +(anon):15> [[ 15 == <1-> ]]
    +(anon):18> typeset -gi '_GITSTATUS_LOCK_FD_POWERLEVEL9K=lock_fd'
    +(anon):20> [[ linux-gnu == cygwin* ]]
    +(anon):41> sysopen -r -o cloexec -u resp_fd /proc/self/fd/14
    +(anon):44> typeset -gi 'GITSTATUS_DAEMON_PID_POWERLEVEL9K=110418'
    +(anon):41> _gitstatus_daemon_p9k_
    +(anon):46> [[ 17 == <1-> ]]
    +(anon):47> typeset -gi '_GITSTATUS_RESP_FD_POWERLEVEL9K=resp_fd'
    +_gitstatus_daemon_p9k_:1> local -i pipe_fd
    +(anon):48> typeset -gi '_GITSTATUS_STATE_POWERLEVEL9K=1'
    +(anon):51> ((  ! async  ))
    +(anon):7> setopt monitor
    +(anon):9> ((  ! _GITSTATUS_STATE_POWERLEVEL9K  ))
    +(anon):51> ((  ! async  ))
    +(anon):52> ((  _GITSTATUS_CLIENT_PID_POWERLEVEL9K == sysparams[pid]  ))
    +(anon):54> local pgid
    +(anon):55> ((  0 < 20  ))
    +(anon):56> [[ -t 17 ]]
    +(anon):57> sysread -s 20 -t 10.0000000000 -i 17 'pgid[$#pgid+1]'
    +(anon):55> ((  20 < 20  ))
    +(anon):59> [[ '              110418' == \ #<1-> ]]
    +(anon):60> typeset -gi 'GITSTATUS_DAEMON_PID_POWERLEVEL9K=pgid'
    +(anon):62> sysopen -w -o cloexec -u req_fd -- /tmp/spin/gitstatus.POWERLEVEL9K.1000.110281.1712129070.1.fifo
    +(anon):62> return 130
    (anon):sysopen:62: can't open file /tmp/spin/gitstatus.POWERLEVEL9K.1000.110281.1712129070.1.fifo: interrupt

  Daemon log (/tmp/spin/gitstatus.POWERLEVEL9K.1000.110281.1712129070.1.daemon.log):

    +_gitstatus_daemon_p9k_:3> local pgid=110418
    +_gitstatus_daemon_p9k_:4> [[ 110418 == <1-> ]]
    +_gitstatus_daemon_p9k_:5> cd -q /
    +_gitstatus_daemon_p9k_:90> ((  lock_fd == -1  ))
    +_gitstatus_daemon_p9k_:9> trap '' PIPE
    +_gitstatus_daemon_p9k_:11> local uname_sm
    +_gitstatus_daemon_p9k_:93> zsystem flock -- /tmp/spin/gitstatus.POWERLEVEL9K.1000.110281.1712129070.1.lock
    +_gitstatus_daemon_p9k_:12> uname_sm=+_gitstatus_daemon_p9k_:12> uname -sm
    +_gitstatus_daemon_p9k_:12> uname_sm='linux x86_64' 
    +_gitstatus_daemon_p9k_:13> [[ 'linux x86_64' == [^\ ]##\ [^\ ]## ]]
    +_gitstatus_daemon_p9k_:14> local uname_s=linux
    +_gitstatus_daemon_p9k_:15> local uname_m=x86_64
    +_gitstatus_daemon_p9k_:17> [[ '' == <1-> ]]
    +_gitstatus_daemon_p9k_:20> local cpus
    +_gitstatus_daemon_p9k_:21> ((  ! 1  ))
    +_gitstatus_daemon_p9k_:21> [[ linux == linux ]]
    +_gitstatus_daemon_p9k_:23> ((  ! 1  ))
    +_gitstatus_daemon_p9k_:23> cpus=+_gitstatus_daemon_p9k_:23> getconf _NPROCESSORS_ONLN
    +_gitstatus_daemon_p9k_:23> cpus=8 
    +_gitstatus_daemon_p9k_:27> args+=( -t 16 ) 
    +_gitstatus_daemon_p9k_:30> mkfifo -- /tmp/spin/gitstatus.POWERLEVEL9K.1000.110281.1712129070.1.fifo
    +_gitstatus_daemon_p9k_:31> print -rnu 14 -- '              110418'

  System information:

    zsh:      5.8.1
    uname -a: Linux spin 6.5.0-1016-gcp #16~22.04.1-Ubuntu SMP Sat Mar  9 00:58:37 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Seems that it was waiting for mkfifo, or more so, the act of writing to the pipe is blocking. Any tips for further troubleshooting?

[romkatv]

It's a bug in the OS. See https://github.com/romkatv/gitstatus/issues?q=%22Linux+spin%22, or https://github.com/romkatv/gitstatus/issues/386.

[bradfeehan]

Ah, thank you so much for tracking those down. Yes, I'm a user of Isospin, Shopify's internal OS used for our development environments.

It's not public, but since it seems to be an issue with FIFO pipes, maybe you could suggest a simple reproduction of what gitstatusd/powerlevel is doing with FIFOs? I haven't used them before.

I wonder if it's because the /tmp filesystem is mounted as nodev?

$ mount
...
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,inode64)
...

Though I can create one:

$ mkfifo /tmp/foo
$ ls -al /tmp/foo
prw-rw-r-- 1 spin spin 0 Apr  9 13:36 /tmp/foo|

And it seems that writing and reading works:

$ echo lol >> foo & # this blocks waiting for a read
[1] 293438
$ jobs
[1]  + running    echo lol >> foo
$ cat foo
lol
[1]  + 293438 done       echo lol >> foo

---

this bug happens on new shells, but when I interrupt (Ctrl+C), then run exec zsh most of the time everything works fine.

[romkatv]

I wonder if it's because the /tmp filesystem is mounted as nodev?

This should not matter: fifos aren't devices.

Perhaps you can reproduce the problem by replacing the content of your .zshrc with the test code you posted above (the one that creates a fifo) and spawning shells. If that does not help, you can try doing what gitstatus does: opening the fifo for reading with exec </tmp/fifo and for writing with sysopen -w -o cloexec -u fd /tmp/fifo. You'll need to run zmodload zsh/system beforehand to make sysopen available. Try opening the fifo for reading and writing in a different order.

You can also dump the stack of the two processes when they hang and send them to the OS people.