Colors do not update in open terminals

[frankdugan3]

When the colors are changed, open terminals are not automatically updated to the new colors. If I open a new terminal, it gets the updated colors. I'm using wallust and for reference, here is my plugin config:

return {
  'RedsXDD/neopywal.nvim',
  name = 'neopywal',
  lazy = false,
  priority = 1000,
  opts = {
    use_palette = 'wallust',
    terminal_colors = true,
    transparent_background = true,
    styles = {
      comments = { 'italic' },
      conditionals = {},
      loops = {},
      functions = {},
      keywords = {},
      includes = {},
      strings = {},
      variables = {},
      numbers = {},
      booleans = {},
      types = {},
      operators = {},
    },
  },
  init = function()
    vim.cmd.colorscheme 'neopywal-dark'
  end,
}

[RedsXDD]

As far as i am aware, this is a problem within Neovim itself. The terminal colors are set with the global variables vim.g.terminal_color_0 ... vim.g.terminal_color_15. However the terminal doesn't update it's colors if you modify these while the terminal is open. And you can confirm that by either modifying them manually or by changing your Neovim colorscheme while there's a terminal buffer opened.

Ideally Neovim would provide some api function to be able to update the terminal buffer in some way that could help me implement some fix for this within Neopywal itself. However i am not sure if such feature is available.

[eylles]

well, there is a hack with a script and a wrapper to handle this, discussed over this pywal16 PR the scripts can be found in this repo

[RedsXDD]

@eylles I did some testing with the nvim pipe commands, honestly i couldn't get it to fix anything related to the issue.
I could be mistaken somewhere, but as far as i am aware the issue is that vim.g.terminal_color_1 .. 14 are not updated when the terminal window is opened inside neovim and the pywal theme is changed.
That change in the pywal's theme triggers Neopywal's reloader module which internally already runs ":colorscheme neopywal" just like the "nvim --listen /tmp/nvim.pipe" in your script aims to do, and yet the terminal colors i mentioned don't change from their older values when ":colorscheme neopywal" is ran, no matter if it's ran from neopywal itself or from "nvim --server /tmp/nvim.pipe --remote-send ':colorscheme neopywal'".

[RedsXDD]

This picture should show what issue i mean, i changed to a wallust colorscheme where most of the "main" colors are black or gray, then i changed to gruvbox.

Neopywal's reloader worked fine for the top non-terminal part of the neovim window (by that i mean, ":colorscheme neopywal" was ran in the background), and yet the terminal colors are all messed up (except from vim.g.terminal_color_0 which is the background)

[eylles]

Hmmm how about directly sending neovim the new color values of vim.g.terminal_color[0...15] from the nvim-colo-reload script, in a manner like this:

rel_other () {
    nvim_send "let g:terminal_color_0  = '${color0}'"
}

Can't test rn but in like half an hour i will.

[eylles]

have not began testing anything tho...

[RedsXDD]

Hmmm how about directly sending neovim the new color values of vim.g.terminal_color[0...15] from the nvim-colo-reload script, in a manner like this:

rel_other () { nvim_send "let g:terminal_color_0 = '${color0}'" }

Can't test rn but in like half an hour i will.

I tested, the changes are only applied once you close and reopen the terminal

[RedsXDD]

Ok i spent about an hour trying out different solutions but none worked.

I did some research and the only conclusion i came up with is that neovim simply hard codes the colors used by the terminal when opening it and nothing is able to change them unless the terminal is closed and reopened again.

[eylles]

what about simulating the color palette update notifications?

according to https://github.com/ahmedelgabri/dotfiles/commit/edd99c15052cfd0f238b1b54cc57109cb0d630d0#commitcomment-154693128 neovim 0.11 and upwards support a new control sequence that was added to both contour and tmux:

https://neovim.io/doc/user/news-0.11.html#_terminal

https://github.com/contour-terminal/contour/blob/master/docs/vt-extensions/color-palette-update-notifications.md

https://github.com/tmux/tmux/pull/4353

hmmmm looking at vim-lumen one could do a hack with an autocmd and a sigwinch but that would not work better than the dec mode 2031 for a good reason

The SIGWINCH event is fired regularly for other events. For example while resizing the window, SIGWINCH can be emitted many times per second, which causes performance issues due to checking the system dark mode preference multiple times per second

if i understood the PR referenced in: https://github.com/vimpostor/vim-lumen/commit/97157aac9f0d24c144a3defdfe5057ee61e18dcb then a call of setoption('background', 'dark', true) like in: https://github.com/neovim/neovim/pull/31350/files#diff-49225a49c226c2f1b36f966d0178c556e204cdc0b660c80db9e4568e03f6ef99R669 should trigger the necessary callbacks to update the colors of the built in neovim terminal, tho this is only going to work in neovim 0.11 and upwards...

[RedsXDD]

what about simulating the color palette update notifications?

according to ahmedelgabri/dotfiles@edd99c1#commitcomment-154693128 neovim 0.11 and upwards support a new control sequence that was added to both contour and tmux:

https://neovim.io/doc/user/news-0.11.html#_terminal

https://github.com/contour-terminal/contour/blob/master/docs/vt-extensions/color-palette-update-notifications.md

tmux/tmux#4353

I could have misunderstood what you sent, but isn't this all about making Neovim itself update it's colorscheme based on the system's settings for light/dark theme, rather than a way to update the colors of the text within the terminal being emulated inside neovim?

hmmmm looking at vim-lumen one could do a hack with an autocmd and a sigwinch but that would not work better than the dec mode 2031 for a good reason

Pretty confident this is the on the same topic as of the text i wrote above but i might take a look at vim-lumen's source code tomorrow to see if there's anything that could be helpful here.

[eylles]

I could have misunderstood what you sent, but isn't this all about making Neovim itself update it's colorscheme based on the system's settings for light/dark theme, rather than a way to update the colors of the text within the terminal being emulated inside neovim?

the way i understand the description of the feature it should update the colorscheme along the colors of the text within the neovim emulated terminal as well as neovim upon recieving the notification that the colorscheme changed between light and dark, if that works as i understand (which is completely up in the air as i don't got neovim 0.11 to test) then we could use some part of that chain of functionality in an "unintended" manner to update the colors of the neovim emulated terminal.

[RedsXDD]

Well i do have neovim 0.11 but i have no clue how i can test this myself on my setup. I am using wezterm so i guess i need to remotely tell wez to "change to light theme". That should make neovim load the neopywal-light variant automatically, which could be used for testing.

But the terminal doc page https://neovim.io/doc/user/news-0.11.html#_terminal states that the terminal sends the theme updates when "background" is changed and DEC 2031 is enabled (which i have no idea what it is), so i guess changing the "background" option manually could work?

[eylles]

i'd say you probably only need to worry about neovim thinking it has dec 2031 enabled and perhaps sending first the text color as background and then the actual background color from nvim-colo-reload could work