[Bug]: `wezterm` CLI commands seem to run through shell, but it shouldn't

[mister-choo]

Similar Issues

• [X] Before filing, I have searched for similar issues.

Neovim Version

NVIM v0.10.2 Build type: Release LuaJIT 2.1.1731601260

Multiplexer Integration

Wezterm

Multiplexer Version

wezterm version: 20241119-101432-4050072d x86_64-unknown-linux-gnu Window Environment: X11 GNOME Shell Lua Version: Lua 5.4

Steps to Reproduce

1. Open nvim
2. Create a wezterm split
3. Navigate to split with direction keys

Expected Behavior

Navigation from wezterm pane to nvim pane to take the same amount of time as from nvim pane to wezterm

Actual Behavior

Navigating from nvim pane to wezterm pane happens with about 2 sec delay

Minimal Configuration to Reproduce

local root = vim.fn.fnamemodify('./.repro', ':p')

-- set stdpaths to use .repro
for _, name in ipairs({ 'config', 'data', 'state', 'cache' }) do
  vim.env[('XDG_%s_HOME'):format(name:upper())] = root .. '/' .. name
end

-- bootstrap lazy
local lazypath = root .. '/plugins/lazy.nvim'
if not vim.loop.fs_stat(lazypath) then
  vim.fn.system({
    'git',
    'clone',
    '--filter=blob:none',
    '--single-branch',
    'https://github.com/folke/lazy.nvim.git',
    lazypath,
  })
end
vim.opt.runtimepath:prepend(lazypath)

-- install plugins
local plugins = {
  -- do not remove the colorscheme! it makes testing nicer
  'folke/tokyonight.nvim',
  'mrjones2014/smart-splits.nvim',
  -- add any other pugins here
}

require('lazy').setup(plugins, {
  root = root .. '/plugins',
})

require('smart-splits').setup({
  -- add any options here
})

vim.keymap.set('n', '<C-Left>', require('smart-splits').move_cursor_left)
vim.keymap.set('n', '<C-Down>', require('smart-splits').move_cursor_down)
vim.keymap.set('n', '<C-Up>', require('smart-splits').move_cursor_up)
vim.keymap.set('n', '<C-Right>', require('smart-splits').move_cursor_right)

-- add anything else here
vim.opt.termguicolors = true
-- do not remove the colorscheme! it makes testing nicer
vim.cmd([[colorscheme tokyonight]])

wezterm config:

local wezterm = require 'wezterm'
local config = {}
if wezterm.config_builder then
    config = wezterm.config_builder()
end

local function is_vim(pane)
    return pane:get_user_vars().IS_NVIM == 'true'
    -- return false
end

local direction_keys = {
    LeftArrow = 'Left',
    DownArrow = 'Down',
    UpArrow = 'Up',
    RightArrow = 'Right',
}

config.leader = { key = "a", mods = "CTRL" }

local function split_nav(key)
    return {
        key = key,
        mods = 'CTRL',
        action = wezterm.action_callback(function(win, pane)
            if is_vim(pane) then
                win:perform_action({ SendKey = { key = key, mods = 'CTRL' }, }, pane)
            else
                win:perform_action({ ActivatePaneDirection = direction_keys[key] }, pane)
            end
        end),
    }
end

config.keys = {
    {
        key = "v",
        mods = "LEADER",
        action = wezterm.action { SplitHorizontal = {
            domain = "CurrentPaneDomain" } }
    },
    split_nav('LeftArrow'),
    split_nav('RightArrow'),
    split_nav('DownArrow'),
    split_nav('UpArrow'),
}

return config

Additional Details and/or Screenshots

No response

[mister-choo]

I found discussions mentioning this issue, but they were all closed. However I found this behavior to be unchanged for about a year now

[mrjones2014]

Like we've done with the other issues, could you test the performance of wezterm cli adjust-pane-size --amount 3 Left? You should be able to do this with the time command.

[mister-choo]

Like we've done with the other issues, could you test the performance of wezterm cli adjust-pane-size --amount 3 Left? You should be able to do this with the time command.

Why adjust-pane-size? I'm not really using it here

[mrjones2014]

Just to narrow down if the issue is the plugin or the wezterm cli, you can try wezterm activate-pane-direction Left instead

[mister-choo]

Ran this script:

set N 100
set total_time 0

for i in (seq 1 $N)
        set start_time (date +%s%N)
        wezterm cli activate-pane-direction Right
        set end_time (date +%s%N)
        set elapsed (math "($end_time - $start_time)/1000000000")
        set total_time (math "$total_time + $elapsed")
end

set average_time (math "$total_time / $N")

echo "Ran $N times."
echo "Average execution time: $average_time seconds."

And this is the output: Ran 100 times. Average execution time: 0.117161 seconds.

[mister-choo]

Sure, maybe wezterm cli executes not as fast as it could, but I think the problem is that on one single move it here is how much wezterm_exec executes:

wezterm_exec: 
{ "wezterm", "cli", "list", "--format", "json" }
wezterm_exec: 
{ "wezterm", "cli", "list", "--format", "json" }
wezterm_exec: 
{ "wezterm", "cli", "list", "--format", "json" }
wezterm_exec: 
{ "wezterm", "cli", "list", "--format", "json" }
wezterm_exec: 
{ "wezterm", "cli", "list", "--format", "json" }
wezterm_exec: 
{ "wezterm", "cli", "activate-pane-direction", "Right" }
wezterm_exec: 
{ "wezterm", "cli", "list", "--format", "json" }

[mister-choo]

Investigated this some more and looks like it's a problem with fish: $ time wezterm cli activate-pane-direction Right

Executed in 122.03 millis fish external usr time 3.97 millis 439.00 micros 3.53 millis sys time 13.34 millis 79.00 micros 13.26 millis

In different shells there isn't a delay and time command add some strange time which isn't usr or sys time

[mrjones2014]

Hmm, I'm on Fish shell as well and have never been able to reproduce the performance issues.

Additionally, according to :h system()

If {cmd} is a List it runs directly (no 'shell').

So the shell shouldn't even be involved.

But, if you want to try it, you can try changing the wezterm_exec function to run the command like bash --noprofile --norc -c "wezterm cli whatever-command-here" instead of directly invoking wezterm.

We could also see if changing vim.fn.system(command) to vim.system(command):wait() makes any difference (looks like that's a newer API that's been added since I wrote this code).

[mister-choo]

Hmm, I'm on Fish shell as well and have never been able to reproduce the performance issues.

Additionally, according to :h system()

If {cmd} is a List it runs directly (no 'shell').

So the shell shouldn't even be involved.

But, if you want to try it, you can try changing the wezterm_exec function to run the command like bash --noprofile --norc -c "wezterm cli whatever-command-here" instead of directly invoking wezterm.

We could also see if changing vim.fn.system(command) to vim.system(command):wait() makes any difference (looks like that's a newer API that's been added since I wrote this code).

Yes, I tested with the new api, but performance was the same Issue was with my startup speed of fish, which I fixed and there's no delay now. However it's still really weird, as it shouldn't be involved at all, from what I investigated I couldn't find the route cause Maybe it's some kind of integration layer thing with wezterm.. but honestly I'm stumped

Now that I'm testing wezterm cli activate-pane-direction also improved after I improved startup time for fish

[mrjones2014]

However it's still really weird, as it shouldn't be involved at all

Yeah, this is the part that I'm still stumped by. I'm passing the command to vim.fn.system(command) as a list, which according to the help docs, means that it shouldn't involve the shell at all.

[mrjones2014]

I wonder if using uv.spawn() would change anything.

[georgeguimaraes]

Thanks, I thought I was going crazy changing my config, thinking it was my problem. Tks for digging into this.

Btw, i'm using wezterm, zsh and Neovim. No tmux. My zsh takes a while to load as well (~1 second)

[georgeguimaraes]

I've removed everything from my .zshrc (compinit, zsh plugins, powerlevel10k, asdf, fzf integration) and smart-splits moved instantly from Neovim to my wezterm pane.

So yeah, it appears that zsh is being called somewhere.

[mrjones2014]

I really do not understand why. According to the Neovim help docs:

{cmd} is treated as in |jobstart()|:
If {cmd} is a List it runs directly (no 'shell').
If {cmd} is a String it runs in the 'shell', like this: >vim
  call jobstart(split(&shell) + split(&shellcmdflag) + ['{cmd}'])

And my code is:

local function wezterm_exec(cmd)
  local command = vim.deepcopy(cmd)
  table.insert(command, 1, config.wezterm_cli_path)
  table.insert(command, 2, 'cli')
  return vim.fn.system(command)
end

It's a list of arguments... why is the shell involved???

[mister-choo]

I really do not understand why. According to the Neovim help docs:

{cmd} is treated as in |jobstart()|:
If {cmd} is a List it runs directly (no 'shell').
If {cmd} is a String it runs in the 'shell', like this: >vim
  call jobstart(split(&shell) + split(&shellcmdflag) + ['{cmd}'])

And my code is:

local function wezterm_exec(cmd)
  local command = vim.deepcopy(cmd)
  table.insert(command, 1, config.wezterm_cli_path)
  table.insert(command, 2, 'cli')
  return vim.fn.system(command)
end

It's a list of arguments... why is the shell involved???

This is probably a wezterm issue. wezterm cli might use shell in some way (weren't able to reproduce). Or observing an environment variable creates slowdown somehow

[georgeguimaraes]

Interesting @mister-choo.

It does seem that wezterm does some stuff finding a GUI to connect to and maybe spawning a shell: https://github.com/wez/wezterm/blob/main/wezterm/src/cli/mod.rs#L168

[georgeguimaraes]

I'm testing folke's approach to navigate between neovim and wezterm. I've disable smart-splits keymaps and I'm using this:

function M.wezterm()
  local nav = {
    h = "Left",
    j = "Down",
    k = "Up",
    l = "Right",
  }

  local function navigate(dir)
    return function()
      local win = vim.api.nvim_get_current_win()
      vim.cmd.wincmd(dir)
      local pane = vim.env.WEZTERM_PANE
      if vim.system and pane and win == vim.api.nvim_get_current_win() then
        local pane_dir = nav[dir]
        vim.system({ "wezterm", "cli", "activate-pane-direction", pane_dir }, { text = true }, function(p)
          if p.code ~= 0 then
            vim.notify(
              "Failed to move to pane " .. pane_dir .. "\n" .. p.stderr,
              vim.log.levels.ERROR,
              { title = "Wezterm" }
            )
          end
        end)
      end
    end
  end

  -- Move to window using the movement keys
  for key, dir in pairs(nav) do
    vim.keymap.set("n", "<C-" .. key .. ">", navigate(key), { desc = "Go to " .. dir .. " window" })
  end
end

return M

So far, so good, no hanging up, even with my bloated zsh.

Not sure the difference in using vim.system instead of vim.fn.system.

Folke's code: https://github.com/folke/dot/blob/cb1d6f956e0ef1848e57a57c1678d8635980d6c5/nvim/lua/util/init.lua#L109

[mrjones2014]

It could be the difference between vim.fn.system() and vim.system(). I'll put up a PR for yall to test.

[mrjones2014]

Can yall please test #283 and see if that helps

[georgeguimaraes]

Unfortunately, #283 still didn't do it.

To debug it furter, I've modified wezterm_exec to this:

local function wezterm_exec(cmd)
  if vim.fn.executable(config.wezterm_cli_path) == 0 then
    error(string.format('`%s` is not executable', config.wezterm_cli_path))
  end
  local command = vim.deepcopy(cmd)
  table.insert(command, 1, config.wezterm_cli_path)
  table.insert(command, 2, 'cli')
  vim.print(vim.inspect(command))

  local start_time = vim.loop.hrtime()
  local result = vim.system(command, { text = true }):wait()
  local end_time = vim.loop.hrtime()
  local duration_ms = (end_time - start_time) / 1e6

  vim.print(string.format('Command took %.2fms', duration_ms))

  if result.code == 0 then
    return result.stdout
  else
    return result.stderr
  end
end

Now, the first time I hit C-j, I get this:

{ "wezterm", "cli", "list", "--format", "json" }
Command took 292.81ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 274.99ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 269.63ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 265.17ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 271.35ms
{ "wezterm", "cli", "activate-pane-direction", "Down" }
Command took 16.82ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 14.24ms

If I navigate it again to my Neovim pane and hit C-j a second time, I get much better results:

{ "wezterm", "cli", "list", "--format", "json" }
Command took 16.72ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 19.31ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 14.40ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 14.89ms
{ "wezterm", "cli", "activate-pane-direction", "Down" }
Command took 14.14ms
{ "wezterm", "cli", "list", "--format", "json" }
Command took 14.61ms

So the problem seems to be that wezterm itself takes a long time to run wezterm cli list --format json for the first time, but to be honest whenever I run it in a zsh pane, it runs instantly regardless.

[curbol]

I was also having delay (1-3 seconds) when moving from nvim to wezterm. I found wezterm-move.nvim which I'm using for only the one case of moving from nvim to wezterm. I'm using smart-splits for everything else and it's much better. There's still a small delay, but it's less than 1 second so I can live with that.

I haven't looked into the source code yet, but thought it might be useful for others investigating the issue.

For more context, I use LazyVim and here's some system info:

OS: macOS 15.1.1 24B91 arm64
Shell: zsh 5.9
Terminal: WezTerm

[muellerj]

I ran into a related problem today, where I had downloaded a build of WezTerm from Github on a new machine and subsequently wezterm was not in my $PATH. Having the following lines as part of wezterm_exec would have helped:

if vim.fn.executable(config.wezterm_cli_path) == 0 then
  error(string.format('`%s` is not executable', config.wezterm_cli_path)) 
end

[jeeeem]

I created this issue last 2 yrs ago on wezterm repo, but still experiencing this issue. I just don't have enough time from last time to thoroughly investigate why this is happening, hoping it fix soon tho to whichever has the issue, but I think the wezterm cli has the issue from what I remember when I last tested it. https://github.com/wezterm/wezterm/issues/3597

related issue: https://github.com/wezterm/wezterm/issues/6436 https://github.com/wezterm/wezterm/issues/5548

[mrjones2014]

Yes, unfortunately Wezterm development pace has become glacial which is a big reason I moved onto Ghostty+Zellij

[michaelfortunato]

I am getting this issue as well with kitty. That is, fast from kitty to neovim, slow from neovim to kitty.

[michaelfortunato]

This only happens when I am on remote. OK. At least for Kitty I think I know why. @mrjones2014

In the case of going from neovim to terminal emulator: move_cursor_left calls mux.move_pane which calls move_multiplexer_inner which calls get_current_pane_id. For kitty its get_current_pane_id which is slow, because it calls kitten @ ls, and searches through tables. The current pane id is not used to actually move the cursor, which is my use case, but rather to as a way to check if the move was successful. if current_pane ~= new_pane...

I think for kitty get_current_pane_id could be made more efficient by having kitty filter the calls via kitten @ ls --match-tab state:active.

In general, I do not think that we need to get the pane (window in kitty parlance) id if we are moving windows, at least on kitty.

Finally, another significant speed up for me was avoiding the call to the custom kitten neighboring_window.py. As I advocated for in the past (but disappeared on it in March I am sorry), we should avoid using python kittens when we can, as starting the python interpreter for each command is slow. Instead, we should (and I do in my monkeypatched config), replace the line at

https://github.com/mrjones2014/smart-splits.nvim/blob/f46b79cf9e83b0a13a4e3f83e3bd5b0d2f172bf2/lua/smart-splits/mux/kitty.lua#L101 with

local ok, _ = pcall(vim.fn.system, { 'kitten', '@', 'focus-window', '--match', 'neighbor:' .. direction })

Notice that doing so avoids a call to neighboring_window.py so we do not have to start up python for this command, and simplifies the dependencies (gets rid of neighboring_window.py).

Note: The issue with swapping neighboring_window.py kitten for calls to kitten @ focus-window ... is that TMUX users will be left out.

Another thing that significantly sped up the process for me was avoiding the call to mux.get() in api.lua's call to mux.move_pane().

First, https://github.com/mrjones2014/smart-splits.nvim/blob/f46b79cf9e83b0a13a4e3f83e3bd5b0d2f172bf2/lua/smart-splits/api.lua#L377 Which calls the expensive part https://github.com/mrjones2014/smart-splits.nvim/blob/f46b79cf9e83b0a13a4e3f83e3bd5b0d2f172bf2/lua/smart-splits/mux/init.lua#L75

And this function is expensive because it calls out to a lua dynamically.

https://github.com/mrjones2014/smart-splits.nvim/blob/f46b79cf9e83b0a13a4e3f83e3bd5b0d2f172bf2/lua/smart-splits/mux/init.lua#L46

---

There are trade offs to this dynamic loading, but I think we should ideally configure the terminal multiplexer and do expensive syscalls like check for multiplexer compatibility via environment variables (ie $KITTY_LISTEN_ON) only once at plugin startup in setup(). This avoids the call in M.get() https://github.com/mrjones2014/smart-splits.nvim/blob/f46b79cf9e83b0a13a4e3f83e3bd5b0d2f172bf2/lua/smart-splits/mux/init.lua#L55 which I think is responsible for the lag.

[mrjones2014]

Very interesting, and thanks for the detailed investigation!

Would you be willing to submit a PR for the Kitty parts?

I can put up a PR to avoid having to redo the expensive dynamic Lua and environment lookups.

[michaelfortunato]

I can't guarantee I have time at this point because I do not know how to translate the tmux logic in neighboring_window.py to the kitten command line. I definitely want to though in spirit. I'm sorry

[mrjones2014]

@michaelfortunato could you test #360? the neighboring_window.py is specific to Kitty, it is not used in tmux, so that doesn't matter.

[mrjones2014]

I think this might be resolved by #383

Please comment if not.