syntax highlighting not working with zsh 5.9 and zsh-vi-mode

[iteemhe]

After updating to the latest zsh version. The zsh-syntax-highlighting does not work anymore.

OS: Archlinux
Version: zsh 5.9 (x86_64-pc-linux-gnu)

---

Related Config:

plugins=(
    zsh-autosuggestions
    zsh-syntax-highlighting
    # vi-mode
    zsh-vi-mode
    git
    # forgit
    # sudo
    # macos
    web-search
    # copydir
    # copyfile
    # copybuffer
    dirhistory
    colored-man-pages
    # vscode
    # xcode
)

[mikkksewce]

Same issue. Also on Arch. Updated zsh to 5.9 a few hours ago. Not using ohmyzsh or anything. Haven't changed anything config related. Also using dracula zsh-syntax-highlighting theme

ZSH_HIGHLIGHT_HIGHLIGHTERS=(main cursor)
typeset -gA ZSH_HIGHLIGHT_STYLES

ZSH_HIGHLIGHT_STYLES[builtin]='fg=#8BE9FD'
ZSH_HIGHLIGHT_STYLES[reserved-word]='fg=#8BE9FD'
ZSH_HIGHLIGHT_STYLES[hashed-command]='fg=#8BE9FD'

ZSH_HIGHLIGHT_STYLES[commandseparator]='fg=#FF79C6'
ZSH_HIGHLIGHT_STYLES[command-substitution-delimiter]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[command-substitution-delimiter-unquoted]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[process-substitution-delimiter]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[back-quoted-argument-delimiter]='fg=#FF79C6'
ZSH_HIGHLIGHT_STYLES[back-double-quoted-argument]='fg=#FF79C6'
ZSH_HIGHLIGHT_STYLES[back-dollar-quoted-argument]='fg=#FF79C6'

ZSH_HIGHLIGHT_STYLES[command-substitution-quoted]='fg=#F1FA8C'
ZSH_HIGHLIGHT_STYLES[command-substitution-delimiter-quoted]='fg=#F1FA8C'
ZSH_HIGHLIGHT_STYLES[single-quoted-argument]='fg=#F1FA8C'
ZSH_HIGHLIGHT_STYLES[single-quoted-argument-unclosed]='fg=#FF5555'
ZSH_HIGHLIGHT_STYLES[double-quoted-argument]='fg=#F1FA8C'
ZSH_HIGHLIGHT_STYLES[double-quoted-argument-unclosed]='fg=#FF5555'
ZSH_HIGHLIGHT_STYLES[rc-quote]='fg=#F1FA8C'

ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[dollar-quoted-argument-unclosed]='fg=#FF5555'
ZSH_HIGHLIGHT_STYLES[dollar-double-quoted-argument]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[assign]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[named-fd]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[numeric-fd]='fg=#F8F8F2'

ZSH_HIGHLIGHT_STYLES[unknown-token]='fg=#FF5555'
ZSH_HIGHLIGHT_STYLES[path]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[path_pathseparator]='fg=#FF79C6'
ZSH_HIGHLIGHT_STYLES[path_prefix]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[path_prefix_pathseparator]='fg=#FF79C6'
ZSH_HIGHLIGHT_STYLES[globbing]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[history-expansion]='fg=#BD93F9'
#ZSH_HIGHLIGHT_STYLES[command-substitution]='fg=?'
#ZSH_HIGHLIGHT_STYLES[command-substitution-unquoted]='fg=?'
#ZSH_HIGHLIGHT_STYLES[process-substitution]='fg=?'
#ZSH_HIGHLIGHT_STYLES[arithmetic-expansion]='fg=?'
ZSH_HIGHLIGHT_STYLES[back-quoted-argument-unclosed]='fg=#FF5555'
ZSH_HIGHLIGHT_STYLES[redirection]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[arg0]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[default]='fg=#F8F8F2'
ZSH_HIGHLIGHT_STYLES[cursor]='fg=#FFFFFF'

source $HOME/.config/zsh/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh

[iteemhe]

Here is an alternative for zsh-syntax-highlight. It works fine on my machine.

https://github.com/zdharma-continuum/fast-syntax-highlighting

I guess the issue is casued by the deprecated zsh funcitons in version 5.9. There is a compatability section in 5.9 release note.

[phy1729]

I'm running on 5.9 without issues. Since you're also running z-asug, the issue may be related to zsh-users/zsh-autosuggestions#529 . Can you see if syntax highlighting returns with z-asug commented out?

[CRY-TeX]

I'm also running zsh 5.9. For me syntax highlighting works fine until I add zsh-vi-mode to the plugins list in my zshrc. (Using oh-my-zsh plugins). Before zsh 5.9 both zsh-syntax-highlighting and zsh-vi-mode worked like a charm together.

[TheOmegaCarrot]

I am also getting this same issue, also on up-to-date Arch.

plugins=(
	zsh-syntax-highlighting
	zsh-autosuggestions
	zsh-vi-mode
	sudo
	command-not-found
)

Commenting out zsh-autosuggestions did not fix it for me. Commenting out zsh-vi-mode did work.

[danielshahaf]

Are y'all running the latest z-sy-h master? If so, say so explicitly. If not, upgrade and try again.

Excuse brevity. Will reply more later.

[CRY-TeX]

Yes running the latest version on master installed with the oh-my-zsh installation directions (commit id "caa749d030d22168445c4cb97befd406d2828db0"). For me it is still broken.

[kstolp]

I'm using the latest commits for both plugins, and am experiencing the same bug. zsh-syntax-highlighting = caa749d zsh-vi-mode = 9e909d0

If I set the zsh-vi-mode option, ZVM_INIT_MODE='sourcing', then syntax highlighting works again.

Hopefully this help debug.

[danielshahaf]

@kstolp Thanks a lot.

I'm going to tentatively assume that everyone here uses the zsh-vi-mode plugin. (The third-party plugin linked above; not merely the builtin bindkey -v functionality.) If that's not the case for anyone, speak up.

I suspect the problem has to do with how the zsh-vi-mode plugin's use of zle-line-pre-redraw, since that widget is used by default by z-sy-h under zsh 5.9 but not (by default) under older zsh's.

There seem to be several problems with the zsh-vi-mode plugin's approach:

1. zvm_define_widget() lacks ${(q)}'s on its eval.
2. zvm_zle-line-pre-redraw() doesn't inspect $rawfunc (set by zvm_widget_wrapper(), which zvm_define_widget() arranges to be called), so any pre-redraw hooks installed prior to zsh-vi-mode aren't run at all. (They probably shouldn't fix that by changing those functions I just named but by removing those functions altogether and using add-zle-hook-widget instead.)
3. That also goes for the zsh-vi-mode plugin's zle-line-init and zle-line-finish wrappers, at least.
4. zvm_zle-line-pre-redraw() calls zvm_update_highlight() which, if I'm reading its code correctly, could remove some $region_highlight entries from plugins other than itself. (Note that multiple highlights can sometimes combine; e.g., region_highlight=("0 5 fg=red" "0 5 bold,underline").) It should use the memo= feature of zsh 5.9 to only remove its own entries. (Note that z-sy-h doesn't use redrawhook on zsh versions that lack the memo= feature precisely in order to avoid breaking other plugins.)

The most important of these is (2) which, if I'm reading the zsh-vi-mode plugin's correctly, causes that plugin to trample on plugins installed prior to itself.

I don't see anything z-sy-h can do about a plugin that runs after z-sy-h's init code and arranges for z-sy-h's add-zle-hook-widget-registered hook not to be called.

/cc @jeffreytse

[danielshahaf]

Perhaps the following? I don't have an environment to test it in:

diff --git a/zsh-vi-mode.zsh b/zsh-vi-mode.zsh
index c8cce0b..abea50f 100644
--- a/zsh-vi-mode.zsh
+++ b/zsh-vi-mode.zsh
@@ -3253,7 +3253,8 @@ function zvm_init() {
   zvm_define_widget zvm_switch_keyword
 
   # Override standard widgets
-  zvm_define_widget zle-line-pre-redraw zvm_zle-line-pre-redraw
+  autoload add-zle-hook-widget
+  add-zle-hook-widget zle-line-pre-redraw zvm_zle-line-pre-redraw
 
   # Ensure the correct cursor style when an interactive program
   # (e.g. vim, bash, etc.) starts and exits

[danielshahaf]

Here is an alternative for zsh-syntax-highlight. It works fine on my machine.

https://github.com/zdharma-continuum/fast-syntax-highlighting

FWIW there's been a proposal to merge that back into this project. See https://github.com/zdharma-continuum/fast-syntax-highlighting/issues/8

I guess the issue is casued by the deprecated zsh funcitons in version 5.9. There is a compatability section in 5.9 release note.

Deprecations wouldn't cause regressions; incompatible changes would be required for that.

The one entry in zsh's list of incompatibilities [in its README file] that's related to z-sy-h is the region_highlight change. It's possible we could see 5.9-only bugs caused by the zsh_highlight__memo_feature probe starting to succeed. I don't think that's what's actually happening here (see above), but still, good call.

[jhony75]

Don't know if this might be the cause for someone as well but, I solved this issue by copying the zsh-syntax-highlight file from this repo to my machine.

As you can see, apparently there was some random code, I don't really know why. I do not know how to see the version of the plugin, but will be happy to update this comment if someone tells me how.

164a165
> <<<<<<< Updated upstream
169a171,176
> =======
>         # On zsh version 5.8.0.2 between the aforementioned commit and the
>         # first Config/version.mk bump after it (which, at the time of writing,
>         # is yet to come), this condition will false negative.
>         if is-at-least 5.9.0.0 $ZSH_VERSION.0.0; then
> >>>>>>> Stashed changes
417c424
< #    function — a catch-22.
---
> #    function — a catch-22.
418a426
> <<<<<<< Updated upstream
420a429,433
> =======
> #    See _zsh_highlight for the magic version number.  (The use of 5.8.0.2
> #    rather than 5.8.0.3 as in the _zsh_highlight is deliberate.)
> if is-at-least 5.9.0.0 $ZSH_VERSION.0.0 && _zsh_highlight__function_callable_p add-zle-hook-widget
> >>>>>>> Stashed changes

The information that I know is that I use the plugin with oh-my-zsh and my system is Arch Linux 5.17.9

This is the first time that I try to do something like this, so if it's wrong or I can do something better, I'll be happy to do it.

[danielshahaf]

@jhony75 Let's break this down:

Don't know if this might be the cause for someone as well but, I solved this issue by copying the zsh-syntax-highlight file from this repo to my machine.

In general, replacing a single file is not guaranteed to work, since the various files may have interdependencies.

As you can see, apparently there was some random code, I don't really know why. I do not know how to see the version of the plugin, but will be happy to update this comment if someone tells me how.

typeset -p ZSH_VERSION ZSH_PATCHLEVEL ZSH_HIGHLIGHT_VERSION ZSH_HIGHLIGHT_REVISION. If that prints HEAD, then run git rev-parse HEAD in the plugin's directory as well (see #863).

164a165
> <<<<<<< Updated upstream
169a171,176
> =======
>         # On zsh version 5.8.0.2 between the aforementioned commit and the
>         # first Config/version.mk bump after it (which, at the time of writing,
>         # is yet to come), this condition will false negative.
>         if is-at-least 5.9.0.0 $ZSH_VERSION.0.0; then
> >>>>>>> Stashed changes
417c424
< #    function — a catch-22.
---
> #    function — a catch-22.
418a426
> <<<<<<< Updated upstream
420a429,433
> =======
> #    See _zsh_highlight for the magic version number.  (The use of 5.8.0.2
> #    rather than 5.8.0.3 as in the _zsh_highlight is deliberate.)
> if is-at-least 5.9.0.0 $ZSH_VERSION.0.0 && _zsh_highlight__function_callable_p add-zle-hook-widget
> >>>>>>> Stashed changes

OK. First of all, the is-at-least 5.9.0.0 must be from https://github.com/zsh-users/zsh-syntax-highlighting/issues/857#issuecomment-1038295881, as no other place in this repository has 5.9.0.0 here.

The diff posted shows that the second file (the bar in diff foo bar) had conflict markers. So, you probably applied the patch in the linked comment, then updated (pulled), which caused a text conflict, which you "resolved" by manually downloading the conflicted file. This would "work" for now, but you won't be able to pull any further changes to that file to your worktree. (The fetch would succeed but the fast-forward merge wouldn't.)

You probably need to resolve the conflict now. Run git status without arguments in z-sy-h's directory and it should tell you what to do. If that doesn't work, you can always clone the z-sy-h repository anew. (It's possible to solve conflicts without re-cloning, of course, but the incantations for that are more likely to cause inadvertent data loss if used incorrectly.)

The information that I know is that I use the plugin with oh-my-zsh and my system is Arch Linux 5.17.9

This is the first time that I try to do something like this, so if it's wrong or I can do something better, I'll be happy to do it.

You'll almost always want to run diff -u foo bar rather than merely diff foo bar (unified diffs are the de facto standard, and they include context which helps both human readers and patch(1)), and git diff rather than diff -u foo bar when the files being diffed are in git. (Nevertheless, there's no need to post the above diff again.)

[kuang22]

Met this problem on MacOS too.

[mohkale]

I've got this problem as well with zsh 5.9 (x86_64-pc-linux-gnu) but I'm not using zsh-vi-mode. I've only got 3 plugins setup (and some color customizations) but even just using zsh-syntax-highlighting by itself I'm still not getting any syntax highlighting anymore (since about a month back). Strangely zdharma-continuum/fast-syntax-highlighting does still work.

[yuze1216]

I have the same problem. On my Mac, the version of Zsh is zsh 5.8.1 (x86_64-apple-darwin21.0)。 I tried almost everything I could find, but Zsh syntax highlighting still didn't work

[danielshahaf]

@mohkale You don't have zsh-vi-mode so you're seeing a different issue. Open a new ticket please and continue there. Post there a minimal reproduction recipe please (without zgen; just source manually as few plugins and other settings as needed to reproduce the problem).

[danielshahaf]

@yuze1216 z-sy-h won't use the redrawhook code under 5.8.1 by default, so you're probably seeing another issue. Open a new ticket please with the details requested in my comment to @mohkale just now.

[yuze1216]

Thank you very much. I think I know the problem. It's my fault. Thank you

[SantinoKeupp]

Perhaps the following? I don't have an environment to test it in:

diff --git a/zsh-vi-mode.zsh b/zsh-vi-mode.zsh
index c8cce0b..abea50f 100644
--- a/zsh-vi-mode.zsh
+++ b/zsh-vi-mode.zsh
@@ -3253,7 +3253,8 @@ function zvm_init() {
   zvm_define_widget zvm_switch_keyword
 
   # Override standard widgets
-  zvm_define_widget zle-line-pre-redraw zvm_zle-line-pre-redraw
+  autoload add-zle-hook-widget
+  add-zle-hook-widget zle-line-pre-redraw zvm_zle-line-pre-redraw
 
   # Ensure the correct cursor style when an interactive program
   # (e.g. vim, bash, etc.) starts and exits

It seems to work for me.

[fbearoff]

Perhaps the following? I don't have an environment to test it in:

diff --git a/zsh-vi-mode.zsh b/zsh-vi-mode.zsh
index c8cce0b..abea50f 100644
--- a/zsh-vi-mode.zsh
+++ b/zsh-vi-mode.zsh
@@ -3253,7 +3253,8 @@ function zvm_init() {
   zvm_define_widget zvm_switch_keyword
 
   # Override standard widgets
-  zvm_define_widget zle-line-pre-redraw zvm_zle-line-pre-redraw
+  autoload add-zle-hook-widget
+  add-zle-hook-widget zle-line-pre-redraw zvm_zle-line-pre-redraw
 
   # Ensure the correct cursor style when an interactive program
   # (e.g. vim, bash, etc.) starts and exits

It seems to work for me.

This fixes the issue for me too,

[mohkale]

Sorry to co-opt this issue, I fixed the unrelated issue I described above and didn't think it made sense to make a new issue just to close it. In my case I had zsh-users/zsh-autosuggestions setup after zsh-users/zsh-syntax-highlighting and that was causing the issue. The FAQ mentioned zsh-syntax-highlighting should be sourced last and moving it to the end of the list of plugins I setup fixed it for me. Don't know why this wasn't an issue before (I suspect zsh-autosuggestions changed how they interact with zle and it inadvertently broke my zsh-syntax-highlighting setup).

[danielshahaf]

It seems to work for me.

This fixes the issue for me too,

Thanks for reporting back. As the patch is to zsh-vi-mode, could someone open an issue or PR there, please, and cross-link this issue and that one?

[danielshahaf]

@mohkale Thanks for following up. I'll say no more as it's an unrelated issue :)

[gabrieldlima]

Perhaps the following? I don't have an environment to test it in:

diff --git a/zsh-vi-mode.zsh b/zsh-vi-mode.zsh
index c8cce0b..abea50f 100644
--- a/zsh-vi-mode.zsh
+++ b/zsh-vi-mode.zsh
@@ -3253,7 +3253,8 @@ function zvm_init() {
   zvm_define_widget zvm_switch_keyword
 
   # Override standard widgets
-  zvm_define_widget zle-line-pre-redraw zvm_zle-line-pre-redraw
+  autoload add-zle-hook-widget
+  add-zle-hook-widget zle-line-pre-redraw zvm_zle-line-pre-redraw
 
   # Ensure the correct cursor style when an interactive program
   # (e.g. vim, bash, etc.) starts and exits

Thank you very much!! That solved the problem. Now I can use zsh-syntax-highlighting and zsh-vi-mode together without any problem.

[danielshahaf]

Triage: issue is in another codebase; nothing we can do on our end to fix it or to work around it; patch for the other project has been devised, tested, and PR'd to that project; thus: closing as "nothing further for us to do". The issue is still reproducible with latest z-sy-h and latest zsh-vi-mode, true; if that affects you, apply the above patch to your copy of zsh-vi-mode.

@fbearoff Thanks for submitting the patch upstream as a PR!

[fabiosilvamonteiro]

enable auto-suggestions based on the history

if [ -f /usr/share/zsh-autosuggestions/zsh-autosuggestions.zsh ]; then . /usr/share/zsh-autosuggestions/zsh-autosuggestions.zsh # change suggestion color ZSH_AUTOSUGGEST_HIGHLIGHT_STYLE='fg=102' fi

replace ZSH_AUTOSUGGEST_HIGHLIGHT_STYLE

to this value:

ZSH_AUTOSUGGEST_HIGHLIGHT_STYLE='fg=102'