"Attempt to shape unibyte text" crash error

[mickeynp]

This issue only occurs on Emacs versions 27.1 and 27.2

Resolution

The quickest fix is to use the master branch of Emacs 27 and compile it yourself (something you are most likely doing anyway due to the slow adoption of new Emacs versions in most distros.)

If you haven't you should try out the native compilation branch also if you haven't.

Details

If you experience this issue on Emacsen newer than Emacs 27.2 when you generate ligatures, then please comment below but include the following:

• Your OS details;
• Build details of Emacs from M-x emacs-version;
• C-h v cairo-version-string;
• C-h v system-configuration-features;
• Your font, and the ligature configuration you are using;
• GUI or Terminal Emacs;
• Whether the issue happens in emacs -q or not.

It's likely a combination of harfbuzz; cairo (or lack thereof); your OS; your font; and the ligatures. Discovering common trends would be useful.

[benley]

• Your OS details;

NixOS 20.09.git.c59ea8b8a0e (Nightingale)

• Build details of Emacs from M-x emacs-version;

GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.21, cairo version 1.16.0)

• C-h v cairo-version-string;

1.16.0

• C-h v system-configuration-features;

"XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER GMP"

• Your font, and the ligature configuration you are using;

PragmataPro Liga Regular, version 0.828

Ligature config:

(use-package ligature
  :config
  (ligature-set-ligatures t '("[ERROR]" "[DEBUG]" "[INFO]" "[WARN]" "[WARNING]"
                              "[ERR]" "[FATAL]" "[TRACE]" "[FIXME]" "[TODO]"
                              "[BUG]" "[NOTE]" "[HACK]" "[MARK]"
                              "# ERROR" "# DEBUG" "# INFO" "# WARN" "# WARNING"
                              "# ERR" "# FATAL" "# TRACE" "# FIXME" "# TODO"
                              "# BUG" "# NOTE" "# HACK" "# MARK"
                              "// ERROR" "// DEBUG" "// INFO" "// WARN" "// WARNING"
                              "// ERR" "// FATAL" "// TRACE" "// FIXME" "// TODO"
                              "// BUG" "// NOTE" "// HACK" "// MARK"
                              "!!" "!=" "!==" "!!!" "!≡" "!≡≡" "!>" "!=<" "#("
                              "#_" "#{" "#?" "#>" "##" "#_(" "%=" "%>" "%>%" "%<%"
                              "&%" "&&" "&*" "&+" "&-" "&/" "&=" "&&&" "&>" "$>"
                              "***" "*=" "*/" "*>" "++" "+++" "+=" "+>" "++=" "--"
                              "-<" "-<<" "-=" "->" "->>" "---" "-->" "-+-" "-\\/"
                              "-|>" "-<|" ".." "..." "..<" ".>" ".~" ".=" "/*" "//"
                              "/>" "/=" "/==" "///" "/**" ":::" "::" ":=" ":≡" ":>"
                              ":=>" ":(" ":-(" ":)" ":-)" ":/" ":\\" ":3" ":D" ":P"
                              ":>:" ":<:" "<$>" "<*" "<*>" "<+>" "<-" "<<" "<<<" "<<="
                              "<=" "<=>" "<>" "<|>" "<<-" "<|" "<=<" "<~" "<~~" "<<~"
                              "<$" "<+" "<!>" "<@>" "<#>" "<%>" "<^>" "<&>" "<?>" "<.>"
                              "</>" "<\\>" "<\">" "<:>" "<~>" "<**>" "<<^" "<!" "<@"
                              "<#" "<%" "<^" "<&" "<?" "<." "</" "<\\" "<\"" "<:" "<->"
                              "<!--" "<--" "<~<" "<==>" "<|-" "<<|" "<-<" "<-->" "<<=="
                              "<==" "=<<" "==" "===" "==>" "=>" "=~" "=>>" "=/=" "=~="
                              "==>>" "≡≡" "≡≡≡" "≡:≡" ">-" ">=" ">>" ">>-" ">>=" ">>>"
                              ">=>" ">>^" ">>|" ">!=" ">->" "??" "?~" "?=" "?>" "???"
                              "?." "^=" "^." "^?" "^.." "^<<" "^>>" "^>" "\\\\" "\\>"
                              "\\/-" "@>" "|=" "||" "|>" "|||" "|+|" "|->" "|-->" "|=>"
                              "|==>" "|>-" "|<<" "||>" "|>>" "|-" "||-" "~=" "~>" "~~>"
                              "~>>" "[[" "]]" "\">" "_|_"))
  (global-ligature-mode t))

Worth noting: If I remove :3 from the ligature list, the error doesn't seem to occur.

• GUI or Terminal Emacs;

GUI

• Whether the issue happens in emacs -q or not.

No, not unless I enable ligature.el features.

[vale981]

* Your OS details;

NixOS unstable

* Build details of Emacs from `M-x emacs-version`;

GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.21, cairo version 1.16.0)

* `C-h v cairo-version-string`;

"1.16.0"

* `C-h v system-configuration-features`;

"XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER GMP"

* Your font, and the ligature configuration you are using;

The font is JetBrains Mono.

(use-package ligature
  :straight (:host github :repo "mickeynp/ligature.el")
  :config
  (ligature-set-ligatures 'prog-mode
                          '("-->" "//" "/**" "/*" "*/" "<!--" ":=" "->>" "<<-" "->" "<-"
                            "<=>" "==" "!=" "<=" ">=" "=:=" "!==" "&&" "||" "..." ".."
                            "|||" "///" "&&&" "===" "++" "--" "=>" "|>" "<|" "||>" "<||"
                            "|||>" "<|||" ">>" "<<" "::=" "|]" "[|" "{|" "|}"
                            "[<" ">]" ":?>" ":?" "/=" "[||]" "!!" "?:" "?." "::"
                            "+++" "??" "###" "##" ":::" "####" ".?" "?=" "=!=" "<|>"
                            "<:" ":<" ":>" ">:" "<>" "***" ";;" "/==" ".=" ".-" "__"
                            "=/=" "<-<" "<<<" ">>>" "<=<" "<<=" "<==" "<==>" "==>" "=>>"
                            ">=>" ">>=" ">>-" ">-" "<~>" "-<" "-<<" "=<<" "---" "<-|"
                            "<=|" "/\\" "\\/" "|=>" "|~>" "<~~" "<~" "~~" "~~>" "~>"
                            "<$>" "<$" "$>" "<+>" "<+" "+>" "<*>" "<*" "*>" "</>" "</" "/>"
                            "<->" "..<" "~=" "~-" "-~" "~@" "^=" "-|" "_|_" "|-" "||-"
                            "|=" "||=" "#{" "#[" "]#" "#(" "#?" "#_" "#_(" "#:" "#!" "#="
                            "&="))
  (global-ligature-mode t))

* GUI or Terminal Emacs;

GUI

* Whether the issue happens in `emacs -q` or not.

Does not unless ligature.el is loaded. Among others // seems to cause problems. It does not happen in pure prog-mode but in conjunction with python-mode, c++-mode...

[vale981]

Interesting: it also happens if the ligature list is completely empty :P and it happens with cascadia code

Related http://emacs.1067599.n8.nabble.com/bug-42522-Emacs-27-100-CPU-when-setting-mode-name-with-ligatures-td515786.html#a517123 and https://www.reddit.com/r/emacs/comments/icem4s/emacs_271_freezes_when_using_font_ligatures/

This particular problem is fixed on emacs master, which will become emacs 28. Either use that, or try patching your emacs with commit fe903c5ab7354b97f80ecf1b01ca3ff1027be446

This commit seems to be related to the modeline.

[vale981]

I can confirm that the commit fixes the issue for me. In nixos you can use:

let
  myEmacs = (pkgs.emacs.override {
    # Use gtk3 instead of the default gtk2
    withGTK3 = true;
    withGTK2 = false;
  }).overrideAttrs (old: {
     patches = (old.patches or []) ++ [
      (pkgs.fetchpatch {
        url = "https://github.com/emacs-mirror/emacs/commit/fe903c5ab7354b97f80ecf1b01ca3ff1027be446.patch";
        sha256 = "0srcgqdc9bhk992vr6yl4vq528fr5414wyz2wf0fxcpw3hz7mhq8";
      })
    ];
  });
in ...

[Alexander-Miller]

Here's something that can be reproduced from emacs -q:

(load "~/.emacs.d/straight/build/ligature/ligature.elc")
(require 'ligature)
(ligature-set-ligatures 't '("=>"))
(ligature-mode t)
(let ((ov (make-overlay 0 1)))
  (overlay-put ov 'after-string "=>"))

Creating the overlay doesn't crash Emacs. Instead it freezes while chugging 100% of a CPU core. kill -12 has no effect, so I can't provide a stack trace.

eros.el creates overlays starting with a =>, so that's where I first encountered this problem, otherwise all the ligatures seem to be fine.

Your OS details;

Linux Manjaro

Build details of Emacs from M-x emacs-version

GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.22, cairo version 1.17.3) of 2020-08-28

C-h v cairo-version-string

1.17.3

C-h v system-configuration-features

XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Your font, and the ligature configuration you are using

The freeze happens with both Fantasque Sans Mono and Source Code Pro.

GUI or Terminal Emacs

GUI

[mickeynp]

@vale981, @Alexander-Miller,

Thank you for your bug reports. It seems this issue's most likely fixed for good in upstream Emacs.

[last-ent]

Turns out the issue was beacuse of pretty-mode.

Once I disabled/removed pretty-mode ligatures works like a charm!

[andyleejordan]

So this is strange. I was experiencing a similar issue where my Emacs was freezing (like in the linked reddit posts). I have HARFBUZZ and CAIRO and Cairo is at 1.16 and I'm using the latest emacs-27 branch on Ubuntu 20.04 with Cascadia Code. I manually bisected the list of ligatures I copied from the readme, and "=>" is the one that caused the freeze for me. Nothing else. Everything else is enabled and working properly, including "==>", which is weird! Sure enough though if I add that single ligature, freeze.

[mickeynp]

@andschwa, I suspect the issue is almost certainly due to a bug in Emacs 27.1. It's fixed in Emacs master (and definitely in the gccemacs branch also) so can you try it again in that one?

[andyleejordan]

I cherry-picked fe903c5ab7354b97f80ecf1b01ca3ff1027be446 on top of emacs-27 branch and that resolved my issue. However, commit 6f955a42198c800f010d43f73ab0a1a0d248efd4 introduced an issue with the example configuration, as "\\" is seen as one character. I removed it and was able to load the example configuration.

[mickeynp]

Hi Andy. Ah interesting. Seems like a bug - probably due to the role of backslash as an escape mechanism in the reader. Thanks for reporting.

[tejasbubane]

Your OS details:

Arch Linux x86_64 (Kernel: 5.10.4-arch2-1)

Build details of Emacs from M-x emacs-version;

27.1

C-h v cairo-version-string;

1.17.3

C-h v system-configuration-features;

"XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP"

Your font, and the ligature configuration you are using;

Fira Code Retina

GUI or Terminal Emacs;

GUI

Whether the issue happens in emacs -q or not.

No

From logs buffer:

Attempt to shape unibyte textError during redisplay: (mode-line-default-help-echo #<window 13 on [redacted-filename].rb>) signaled (quit)
Attempt to shape unibyte text [102 times]
Quit
Attempt to shape unibyte text [100 times]
Quit
Attempt to shape unibyte text [117 times]
Quit
Attempt to shape unibyte text [118 times]
Attempt to shape unibyte textError during redisplay: (mode-line-default-help-echo #<window 13 on [redacted-filename.rb]>) signaled (quit)
Attempt to shape unibyte text [270 times]
Quit
Attempt to shape unibyte text [130 times]
Quit
Attempt to shape unibyte text [72 times]
Quit
Attempt to shape unibyte text [4863 times]
Quit
Attempt to shape unibyte text [53 times]
Quit
Attempt to shape unibyte text [90 times]
Attempt to shape unibyte textError during redisplay: (mode-line-default-help-echo #<window 13 on [redacted-filename.rb]>) signaled (quit)
Attempt to shape unibyte text [76 times]
Quit

[wpcarro]

I spent ~1d troubleshooting why my Emacs was crashing :man_facepalming: (what made things was that I'm using EXWM, so my entire X session crashed). For my own education (and likely that of others) can you share some of the debugging steps you took when troubleshooting this?

In the end I used a combination of edebug-defun and debug-on-entry. But that was after I poured way too much time into other avenues. Any tips?

[wpcarro]

One other question: any tips for unfreezing Emacs when this happens? I tried C-g from within Emacs and the following shell commands (but nothing seemed to work):

pkill -SIGINT emacs
pkill -USR2 emacs
emacsclient --eval '(keyboard-quit)' # also C-g but these shell commands would hang

In the end, I used the nuclear option:

pkill -9 emacs

Is there a better way to interrupt Emacs here and maintain (some of) the application state? If so, that would be a game-changer for my workflow.

[matthew-piziak]

Wow, I've never found a hang that hasn't fallen to SIGUSR2. My condolences!