Single quotes failed in c/c++/php mode since Emacs 26 HEAD-59d0787

[twlz0ne]

Expected behavior

``

Actual behavior

\`\`

Steps to reproduce the problem

$ /path/to/emacs-26_HEAD-fa5e63e -nw -Q -l /path/to/test-smartparens.el

Backtraces if necessary (M-x toggle-debug-on-error)

None

Environment & version information

• smartparens version: smartparens-20170723.1205
• Active major-mode: c/c++/php
• Emacs version (M-x emacs-version): Emacs 26 HEAD-fa5e63e (Since HEAD-59d0787)
• Spacemacs/Evil/Other starterkit (specify which)/Vanilla: none
• OS: macOS 10.11.6
• Configuration (test-smartparens.el):

;;; Usage: /path/to/emacs -nw -Q -l /path/to/test-smartparens.el
(toggle-debug-on-error)

(setq package-user-dir (format "%s/elpa--test-smartparens/%s" user-emacs-directory emacs-version))
(setq package-archives
      '(("gnu" . "https://elpa.gnu.org/packages/")
        ("melpa" . "https://melpa.org/packages/")))

(package-initialize)

(defun require-packages (&rest packages)
  (dolist (pkg packages)
    (unless (package-installed-p pkg)
      (package-refresh-contents)
      (package-install pkg))
    (require pkg)))

(require-packages
 'smartparens
 )

(require 'package)
(defun list-installed-package ()
  (mapcar
   #'car
   (mapcar
    (lambda (p) (cons (package-desc-full-name p) p))
    (delq nil
          (mapcar (lambda (p) (unless (package-built-in-p p) p))
                  (apply #'append (mapcar #'cdr package-alist)))))))

;; ------------------------------------------------------------------

(add-hook 'after-init-hook
          '(lambda ()
             (switch-to-buffer "*.c")
             (insert "// -*- mode: c -*-\n")
             (insert (format "// Installed packages: %s\n" (list-installed-package)))
             (insert "// Press ' below:\n")
             (c-mode)
             (smartparens-mode)
             (execute-kbd-macro "'")
             ))

(run-hooks 'after-init-hook)
;;; test-smartparens.el ends here

[Fuco1]

Thanks for the awesomely detailed repro scenario! Much appreciated.

Can you, as a temporary workaround, try to set sp-escape-quotes-after-insert to nil? I don't have E26 set up at the moment.

[twlz0ne]

@Fuco1 it seems to work well after set sp-escape-quotes-after-insert to nil.

[amatrelan]

Thanks @Fuco1! (setq-default sp-escape-quotes-after-inser nil) worked.

[Miciah]

It appears the issue has been resolved in PHP Mode by recent changes to that mode. However, I am still able to reproduce the issue with CC Mode.

If I open an empty buffer, type M-: (c-mode) RET, type C-q " to enter a literal double-quote, and then type M-: (sp-point-in-string) RET, the function evaluates to t. If I delete the double-quote, type C-q ' to enter a literal single-quote, and then type M-: (sp-point-in-string) RET again, the function evaluates to nil.

Smartparens is using Emacs's syntax parsing function syntax-ppss to determine whether point is inside a comment, code, or string, and Smartparens's sp-escape-quotes-after-insert function uses this context to determine whether to escape newly inserted delimiters. However, CC Mode is extremely strict regarding single-quoted strings (more correctly, character literals): If a single quote is not paired with another single quote, or if the pair does not enclose exactly either a single backslash-escaped character or a single character that is neither a backslash nor another single quote, then the single quote is considered punctuation rather than a string delimiter. Thus an empty pair of single-quotes is parsed as code rather than as a string, which confuses sp-escape-quotes-after-insert, resulting in the spurious escaping.

CC Mode enforces this strict parsing of single quotes by manipulating single-quote characters' syntax-table text property in the c-parse-quotes-after-change function. A mode that is based on CC Mode can enable this strict parsing by including c-parse-quotes-after-change in its c-before-font-lock-functions definition; CC Mode runs the functions in c-before-font-lock-functions from its c-after-change hook in after-change-functions.

Not all CC Mode-based modes enable c-parse-quotes-after-change; of the built-in modes, C, Objective-C, C++, and Java enable it, but AWK, IDL, and Pike do not. PHP Mode used to use Java's definition but now explicitly defines c-before-font-lock-functions to exclude c-parse-quotes-after-change (see https://github.com/emacs-php/php-mode/commit/7e4ba4ec53730172ae003bf6026cc42593b3fdfc, https://github.com/emacs-php/php-mode/commit/40ef9f648da48c7d9571182239b1f322efa520c5, and https://github.com/emacs-php/php-mode/commit/40ef9f648da48c7d9571182239b1f322efa520c5).

So the problem is that syntax-ppss and sp-point-in-string are returning unexpected values for single-quotes in CC Mode. However, I am not sure whether the problem should be fixed in Smartparens or in CC Mode.

[ryanpcmcquen]

I'm getting hit by the same bug: https://github.com/bbatsov/prelude/issues/1233

[Fuco1]

@Miciah Thank you very much for the detailed analysis. I've implemented a fix based on your observations. The issue should be fixed on master now.

[Fuco1]

todo: check that there are regression tests for this issue, also check the error cases from #915