grep-edit-mode does not work well in Embark-exported buffers

[minad]

This issue is a bit special, so I want to write down here what I found for now. The issue only affects Emacs 31, so not yet that relevant.

When exporting grep results from consult-grep, then invoking the new Emacs 31 grep-edit-mode, and then using query-replace, the replacement fails for matches outside of the window (as soon the window scrolls).

The stack trace for occur-after-change-function compiled:

Debugger entered--Lisp error: (wrong-type-argument markerp nil)
  (occur-after-change-function 2524 2527 3)
  (replace-match-maybe-edit "2.3" nil t nil (2524 2527 #<buffer *Embark Export: consult-ripgrep - #2.2*>) nil)
  (perform-replace "2.2" "2.3" t nil nil nil nil nil nil nil nil)
  (query-replace "2.2" "2.3" nil nil nil nil nil)
  (funcall-interactively query-replace "2.2" "2.3" nil nil nil nil nil)
  (command-execute query-replace)

And for occur-after-change-function interpreted:

Debugger entered--Lisp error: (wrong-type-argument markerp nil)
  (marker-buffer nil)
  (let* ((line-beg (line-beginning-position)) (targets (get-text-property line-beg 'occur-target)) (m (occur--targets-start targets)) (buf (marker-buffer m)) col) (if (and (get-text-property line-beg 'occur-prefix) (not (get-text-property end 'occur-prefix))) (progn (if (= length 0) (progn (put-text-property beg end 'occur-target targets) (save-excursion (and (search-forward "\n" end t) (delete-region ... end))))) (let* ((line (- (line-number-at-pos) (line-number-at-pos ...))) (readonly (save-current-buffer (set-buffer buf) buffer-read-only)) (win (or (get-buffer-window buf) (display-buffer buf ...))) (line-end (line-end-position)) (text (save-excursion (goto-char ...) (setq col ...) (buffer-substring-no-properties ... line-end)))) (let ((save-selected-window--state (internal--before-with-selected-window win))) (save-current-buffer (unwind-protect (progn ... ... ... ... ...) (internal--after-with-selected-window save-selected-window--state))))))))
  (save-excursion (goto-char beg) (let* ((line-beg (line-beginning-position)) (targets (get-text-property line-beg 'occur-target)) (m (occur--targets-start targets)) (buf (marker-buffer m)) col) (if (and (get-text-property line-beg 'occur-prefix) (not (get-text-property end 'occur-prefix))) (progn (if (= length 0) (progn (put-text-property beg end 'occur-target targets) (save-excursion (and ... ...)))) (let* ((line (- ... ...)) (readonly (save-current-buffer ... buffer-read-only)) (win (or ... ...)) (line-end (line-end-position)) (text (save-excursion ... ... ...))) (let ((save-selected-window--state ...)) (save-current-buffer (unwind-protect ... ...))))))))
  (occur-after-change-function 2241 2244 3)
  (replace-match-maybe-edit "2.3" nil t nil (2241 2244 #<buffer *Embark Export: consult-ripgrep - #2.2*<2>>) nil)
  (perform-replace "2.2" "2.3" t nil nil nil nil nil nil nil nil)
  (query-replace "2.2" "2.3" nil nil nil nil nil)
  (funcall-interactively query-replace "2.2" "2.3" nil nil nil nil nil)
  (command-execute query-replace)

The problem seems to be that the matches miss occur-target properties for matches outside of the displayed windows. These are usually not present in Grep buffers, but are added by grep-edit--prepare-buffer. Now I confirmed that grep-edit--prepare-buffer fails to add occur-target properties to the matches outside of the window.

Now the odd thing is that when I invoke rgrep, and start grep-edit-mode, then the occur-target properties are added correctly. But not for the Embark exported buffer from consult-grep.

[minad]

Hmm, could it be that the compilation-message properties which are searched for by grep-edit--prepare-buffer to generate the occur-target properties are only added lazily by font locking? This makes sense given that the problem occurs for matches outside of the window. But then I wonder why the normal grep mode immediately adds all the compilation-message properties?!

[minad]

The function compile--ensure-parse makes sure that the compilation-message are generated. This function is called by font locking. Since we do not call compile--ensure-parse anywhere after compilation/grep message insertion, the locations are not generated, only by font locking lazily.

Now it is unclear to me why rgrep adds the properties immediately, if it does after all (I am not yet sure). If the compilation-message properties are supposed to be added lazily anyway, then grep-edit--prepare-buffer would have to first call compile--ensure-parse on the whole buffer. (EDIT: Compilation buffers are usually populated by compilation-filter which indeed calls compile--ensure-parse.)

[minad]

Okay, I proposed a PR with a fix. All it took was a single line. 🥳