Troubleshooting emacs hanging when replying to certain messages with org-msg

[peterhoeg]

I'm looking for assistance on how to trouble-shoot some issues with org-msg.

Certain mails will consistently hang emacs (it starts eating CPU with no way to interrupt it) when replying to them with org-msg. If I do a "regular" reply with mu4e-compose I have no issues. Some senders are more likely to trigger it than others but not not all mails from the offending senders will cause the hang.

I cannot use the profiler to see where it gets stuck as I have no way of interacting with emacs once it hangs.

[yantar92]

One way is setting debug-on-error to t and then sending SIGUSR2 signal to Emacs process when Emacs hangs. As a result, you should be able to get a backtrace of what was running during the hang.

[jsilve24]

I am having a similar issue. Here is a backtrace

Debugger entered--entering a function:
* #f(compiled-function () #<bytecode 0xb89fdae5e24db8a>)()
  font-lock-default-fontify-region(422 1483 nil)
  font-lock-fontify-region(422 1483)
  #f(compiled-function (fun) #<bytecode 0x19b6fa84c1464b3d>)(font-lock-fontify-region)
  jit-lock--run-functions(422 1483)
  jit-lock-fontify-now(422 1922)
  jit-lock-function(422)
  redisplay_internal\ \(C\ function\)()

I am honestly not exactly sure how to go about debugging this but here is censored version of the email that was caussing issues (or at least I am giving the reply temlate I see after SIGUSR2).

References: <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
User-agent: mu4e 1.8.14; emacs 28.2
From: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
To: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Subject: Re: Reviewing Students From Annual Evaluations
Date: Mon, 13 Feb 2023 13:41:11 -0500
In-reply-to: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--text follows this line--
#+OPTIONS: html-postamble:nil num:nil ^:{} toc:nil author:nil email:nil \n:t tex:dvipng eval:nil d:nil
#+STARTUP: hidestars indent inlineimages
:PROPERTIES:
:reply-to: nil
:attachment: nil
:alternatives: (text)
:END:

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx writes:

> Good afternoon Justin,
>
>  
>
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>  
>
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>  
>
> * xxxxxxxxxxxxx
>
> * xxxxxxxxxxxxx
>
> * xxxxxxxx
>
> * xxxxxxx
>
> * xxxxxxxxxxxx
>
> * xxxxxxxx
>
>  
>
>xxxxxxxxxxxxxxxx
>
>xxxxxx
>
> _________________________________________
>
>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
>xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>
> PSU_IST_PMS_287_284-300dp

Note the last item the PSU string at the end seems to be a placeholder for a corporate logo attached to the original email.

[jsilve24]

Update: I have been trying to debug this for a while and I did isolate the issue to the keywords being added to org-font-lock-keywords variable in the org-msg-edit-mode definition.

Specifically, commenting out the setq as in the following code stops emacs from hanging.

That said, it also inhibits font locking so its deffintely not a fix.

(define-derived-mode org-msg-edit-mode org-mode "OrgMsg"
  "Major mode to compose email using Org mode.
Like Org Mode but with these additional/changed commands:
Type \\[org-ctrl-c-ctrl-c] to send the message if the cursor is
  not a C-c C-c Org mode controlled region (Org babel for
  example).
Type \\[org-msg-preview] to preview the final email with
  `browse-url'.
Type \\[message-kill-buffer] to kill the current OrgMsg buffer.
Type \\[message-goto-subject] to move the point to the Subject
  header.
Type \\[org-msg-goto-body] to move the point to the beginning of
  the message body.
Type \\[org-msg-attach] to call the dispatcher for attachment
  commands.

\\{org-msg-edit-mode-map}"
  (setq-local message-sent-message-via nil)
  (add-hook 'message-send-hook 'org-msg-prepare-to-send nil t)
  (add-hook 'message-sent-hook 'undo t t)
  (add-hook 'completion-at-point-functions 'message-completion-function nil t)
  (cond ((message-mail-alias-type-p 'abbrev) (mail-abbrevs-setup))
	((message-mail-alias-type-p 'ecomplete) (ecomplete-setup)))
  (setq org-font-lock-keywords
	(append
	 ;; message-font-lock-keywords
	 org-font-lock-keywords
	 ;; gnus-message-citation-keywords
	 org-msg-font-lock-keywords
	 ))
  (toggle-truncate-lines)
  (org-msg-mua-call 'edit-mode)
  (setq-local kill-buffer-hook 'org-msg-kill-buffer
	      org-link-file-path-type 'absolute)
  (when (featurep 'dnd)
    (setq-local dnd-protocol-alist
                (append org-msg-dnd-protocol-alist dnd-protocol-alist)))
  (unless (= (org-msg-end) (point-max))
    (add-text-properties (1- (org-msg-end)) (point-max) '(read-only t))))

Note that as soon as I try to uncomment messsage-font-lock-keywords or gnus-message-citation-keywords, emacs hangs and I need to SIGUSR2 to regain control.

Within message-font-lock-keywords, I have isolated the issue to the fontification of cited text. If I comment out the following part of that keyword definition then I can included message-font-lock-keywords that I commented out in the minor mode declaration:

   ;; Additional font locks to highlight different levels of cited text
   (let ((maxlevel 1)
         (level 1)
         cited-text-face
         keywords)
     ;; Compute the max level.
     (while (setq cited-text-face
                  (intern-soft (format "message-cited-text-%d" maxlevel)))
       (setq maxlevel (1+ maxlevel)))
     (setq maxlevel (1- maxlevel))
     ;; Generate the keywords.
     (while (setq cited-text-face
                  (intern-soft (format "message-cited-text-%d" level)))
       (setq keywords
             (cons
              `(,(message-font-lock-make-cited-text-matcher level maxlevel)
                (0 ',cited-text-face))
              keywords))
       (setq level (1+ level)))
     keywords)

UPDATE: It gets even weirder, turns out when I first startup emacs and try to reply to one of these problematic messages everything works fine. However, if I load org (by calling org-agenda) and then try to reply to the same message, then I get the pathologic behavior I have been describing above.

Finaly: I am sure its a problem with org-msg as if I disable org-msg then I am able to reply to these emails without problem.

[jeremy-compostella]

Do you have to comment out all of them or only some of them?

On Mon, Feb 13, 2023, 12:49 PM Justin Silverman @.***> wrote:

Update: I have been trying to debug this for a while and I did isolate the issue to the keywords being added to org-font-lock-keywords variable in the org-msg-edit-mode definition.

Specifically, commenting out the setq as in the following code stops emacs from hanging.

That said, it also inhibits font locking so its deffintely not a fix.

(define-derived-mode org-msg-edit-mode org-mode "OrgMsg" "Major mode to compose email using Org mode. Like Org Mode but with these additional/changed commands: Type \[org-ctrl-c-ctrl-c] to send the message if the cursor is not a C-c C-c Org mode controlled region (Org babel for example). Type \[org-msg-preview] to preview the final email with `browse-url'. Type \[message-kill-buffer] to kill the current OrgMsg buffer. Type \[message-goto-subject] to move the point to the Subject header. Type \[org-msg-goto-body] to move the point to the beginning of the message body. Type \[org-msg-attach] to call the dispatcher for attachment commands.

\{org-msg-edit-mode-map}" (setq-local message-sent-message-via nil) (add-hook 'message-send-hook 'org-msg-prepare-to-send nil t) (add-hook 'message-sent-hook 'undo t t) (add-hook 'completion-at-point-functions 'message-completion-function nil t) (cond ((message-mail-alias-type-p 'abbrev) (mail-abbrevs-setup)) ((message-mail-alias-type-p 'ecomplete) (ecomplete-setup))) ;; (setq org-font-lock-keywords ;; (append ;; message-font-lock-keywords ;; ;; org-font-lock-keywords ;; ;; gnus-message-citation-keywords ;; org-msg-font-lock-keywords ;; )) (toggle-truncate-lines) (org-msg-mua-call 'edit-mode) (setq-local kill-buffer-hook 'org-msg-kill-buffer org-link-file-path-type 'absolute) (when (featurep 'dnd) (setq-local dnd-protocol-alist (append org-msg-dnd-protocol-alist dnd-protocol-alist))) (unless (= (org-msg-end) (point-max)) (add-text-properties (1- (org-msg-end)) (point-max) '(read-only t))))

— Reply to this email directly, view it on GitHub https://github.com/jeremy-compostella/org-msg/issues/154#issuecomment-1428560517, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBGXEKVLBV525XRMNPTWXKF5FANCNFSM53N6TOWA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[jsilve24]

Yes, good question. An hour or so of debugging revealed that the problem seems to be in message-font-lock-keywords and gnus-message-citation-keywords. Moreover, it seems to be only in the part of these two that deal with font-lock of quote levels. See my prior comment where I give the section of message-font-lock-keywords that needs to be commented out to make that one work. gnus-message-citation-keywords on the other hand seems to have nothing in it other than code related to quote levels and I foudn that I had to pretty much comment out that entire thing to make it work (at least in my machine).

[jeremy-compostella]

If you have an email you know will hit the issue, could you hit reply (with all the font lock disabled) and save that to a file? The issue should be reproducible on such a file by enabling org-msg-edit-mode on it.

On Mon, Feb 13, 2023, 7:05 PM Justin Silverman @.***> wrote:

Yes, good question. An hour or so of debugging revealed that the problem seems to be in message-font-lock-keywords and gnus-message-citation-keywords. Moreover, it seems to be only in the part of these two that deal with font-lock of quote levels. See my prior comment where I give the section of message-font-lock-keywords that needs to be commented out to make that one work. gnus-message-citation-keywords on the other hand seems to have nothing in it other than code related to quote levels and I foudn that I had to pretty much comment out that entire thing to make it work (at least in my machine).

— Reply to this email directly, view it on GitHub https://github.com/jeremy-compostella/org-msg/issues/154#issuecomment-1429001191, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBCWUXPALNMHTLFQRNLWXLSALANCNFSM53N6TOWA . You are receiving this because you commented.Message ID: @.***>

[jsilve24]

I don't understand org-msg well enough to know if this will work, but here it goes.

From: Justin Silverman <[email protected]>
To: "Foo, Bar" <[email protected]>
Subject: Re: Test
Date: Mon, 13 Feb 2023 21:19:50 -0500
In-reply-to: <CH0PR02MB824241D16A0F5E72DE51133DA2DD9@CH0PR02MB8242.namprd02.prod.outlook.com>
--text follows this line--
#+OPTIONS: html-postamble:nil num:nil ^:{} toc:nil author:nil email:nil \n:t tex:dvipng eval:nil d:nil
#+STARTUP: hidestars indent inlineimages
:PROPERTIES:
:reply-to: ("/tmp/mm-yi7MOB.html" "/tmp/cidYyAWvu")
:attachment: nil
:alternatives: (text html)
:END:


--citation follows this line (read-only)--
"Foo, Bar" <[email protected]> writes:

Test



---

Foo Bar | Graduate Programs Assistant

E103/E105 Westgate Building | [email protected]

University Park | PA | 16802

PSU_IST_PMS_287_284-300dp

Is this what you are looking for?

[jeremy-compostella]

This is a very very simple reply so I got to ask would it have hang Emacs if you did not commented out those font locks ?

On Mon, Feb 13, 2023, 7:22 PM Justin Silverman @.***> wrote:

I don't understand org-msg well enough to know if this will work, but here it goes.

From: Justin Silverman @.> To: "Foo, Bar" @.> Subject: Re: Test Date: Mon, 13 Feb 2023 21:19:50 -0500 In-reply-to: @.***> --text follows this line-- #+OPTIONS: html-postamble:nil num:nil ^:{} toc:nil author:nil email:nil \n:t tex:dvipng eval:nil d:nil #+STARTUP: hidestars indent inlineimages :PROPERTIES: :reply-to: ("/tmp/mm-yi7MOB.html" "/tmp/cidYyAWvu") :attachment: nil :alternatives: (text html) :END:

--citation follows this line (read-only)-- "Foo, Bar" @.***> writes:

Test

---

Foo Bar | Graduate Programs Assistant

E103/E105 Westgate Building | @.***

University Park | PA | 16802

PSU_IST_PMS_287_284-300dp

— Reply to this email directly, view it on GitHub https://github.com/jeremy-compostella/org-msg/issues/154#issuecomment-1429013953, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBDOPK54KGXZ7XK4ZCDWXLT6NANCNFSM53N6TOWA . You are receiving this because you commented.Message ID: @.***>

[jsilve24]

Well S***%. Yes, that email exactly was my test case for most of today. But in the past 2 hours I just updated all my emacs packages and now that email is working fine. Maybe the bug got fixed upstream? Its been hanging around for months now... I will watch for it, if it doesn't show up in the near future I guess my part in this is fixed.

[jeremy-compostella]

That was going to be my next suggestion if you were producing with such a simple email. I would hit the issue way too often to not investigate.

I remember running into some similar issues a very long time ago. I don't remember if I had to fix it myself or not.

Jeremy

On Mon, Feb 13, 2023, 7:36 PM Justin Silverman @.***> wrote:

Well S***%. Yes, that email exactly was my test case for most of today. But in the past 2 hours I just updated all my emacs packages and now that email is working fine. Maybe the bug got fixed upstream? Its been hanging around for months now... I will watch for it, if it doesn't show up in the near future I guess my part in this is fixed.

— Reply to this email directly, view it on GitHub https://github.com/jeremy-compostella/org-msg/issues/154#issuecomment-1429023470, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBCMYZ3CHLG62M5MG2DWXLVTTANCNFSM53N6TOWA . You are receiving this because you commented.Message ID: @.***>

[peterhoeg]

Here are a few of my observations:

1. this happens almost exclusively with mails from a Spanish customer of ours (written in English) using Outlook. Keep in mind that I get mails from people typically writing in Chinese, Korean and Japanese from time to time without any issues.
2. I can only replicate the issue if I have used org-agenda first (which I almost always do).
3. I’ve just tried updating all emacs packages, no change.

[peterhoeg]

I haven't tried finding a definite case that triggers this, but I think it has to do with this scenario:

1. the customer in question has very long disclaimer at the bottom which renders on one line (about 1300 characters)
2. my signature ends up after the disclaimer

If I hit reply on that message, it's initially OK when the disclaimer is further down (I'm guessing emacs is being smart about font-locking and not doing it until it gets into view). If I jump to the very end of the mail, I get the hang. If I then manage to kick emacs enough times to get back control, disable font-locking, delete the signature and enable font-locking again, it's OK.

[jeremy-compostella]

Is there any portion of that signature reproducing the issue if pasted in a org msg buffer that you could share?

On Sun, Feb 19, 2023, 10:48 PM Peter Hoeg @.***> wrote:

I haven't tried finding a definite case that triggers this, but I think it has to do with this scenario:

1. the customer in question has very long disclaimer at the bottom which renders on one line (about 1300 characters)
2. my signature ends up after the disclaimer

If I hit reply on that message, it's initially OK when the disclaimer is further down (I'm guessing emacs is being smart about font-locking and not doing it until it gets into view). If I jump to the very end of the mail, I get the hang. If I then manage to kick emacs enough times to get back control, disable font-locking, delete the signature and enable font-locking again, it's OK.

— Reply to this email directly, view it on GitHub https://github.com/jeremy-compostella/org-msg/issues/154#issuecomment-1436369035, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBGCZ57E36ULTYTA5RDWYMASHANCNFSM53N6TOWA . You are receiving this because you commented.Message ID: @.***>

[peterhoeg]

Is there any portion of that signature reproducing the issue if pasted in a org msg buffer that you could share?

I need to carve out a little bit of time in order to come up with a case that reliably reproduces this.

[JonathanReeve]

I'm having a similar issue when replying to messages. For me, it's all messages, anad it doesn't matter whether I have any signatures set, or what my settings are in org-msg. I'm using Doom Emacs and I suspect there might be something going on there, unless you're not using Doom and you're getting the same issue. I opened an issue here with Doom. Are you getting any freezes related to enabling/disabling word wrap?

[peterhoeg]

It definitely has to do with long lines. Will find more details.

[peterhoeg]

I have not had this issue since I started using this:

(add-hook 'org-msg-edit-mode-hook
          (lambda ()
            (setq org-font-lock-keywords (delq 'gnus-message-citation-keywords org-font-lock-keywords))))