org-msg icon indicating copy to clipboard operation
org-msg copied to clipboard

everything below --citation follows this line (read-only)-- erased

Open jonathanwilner opened this issue 3 years ago • 74 comments

Every time I try to reply to a message using org-msg, any text that should be cited is instead removed from the message. This is true in the preview & the final results.

System is Notmuch on Emacs 27.1 or Emacs 28 on Mac OS X Catalina. I'm using Doom Emacs. I've tried the latest version from master as well as the experimental branch. Both produce the same results.

I'm running the latest notmuch from git.

Any ideas?

jonathanwilner avatar Sep 30 '20 23:09 jonathanwilner

I believe this is the intended behaviour. If you want to quote something in your reply you should specifically include it above the --citation...-- line in a #+begin_quote ... #+end_quote block

TimQuelch avatar Oct 01 '20 00:10 TimQuelch

So, it’s cited below the original email in order to allow the user to copy the text into the right block ? That makes sense.

Is there an example configuration that uses org-msg only for composition, not for replies, In order to still use html mail, but make it less cumbersome to keep threads full of info for others in the email conversation?

jonathanwilner avatar Oct 01 '20 14:10 jonathanwilner

Is there an example configuration that uses org-msg only for composition, not for replies, In order to still use html mail, but make it less cumbersome to keep threads full of info for others in the email conversation?

Could you be more specific ? I am having a hard understanding your request

jeremy-compostella avatar Dec 22 '20 01:12 jeremy-compostella

@jeremy-compostella Sorry to intrude. But I was wondering the exact same question when trying to use org-msg with notmuch.

With mu4e, when replying to an email, the original email appears below my reply(below a grey horizontal line), making my reply totally Outlook style.

But with notmuch, my reply would only contain my reply, without the original email below it. (Hence as this issue said, things below --citation follows...--- are erased).

I could not figure out why there is such difference. Would you mind sharing your thoughts? Please let me know in case my description above isn't clear.

whudwl avatar May 04 '21 09:05 whudwl

@jeremy-compostella Sorry to intrude. But I was wondering the exact same question when trying to use org-msg with notmuch.

With mu4e, when replying to an email, the original email appears below my reply(below a grey horizontal line), making my reply totally Outlook style.

But with notmuch, my reply would only contain my reply, without the original email below it. (Hence as this issue said, things below --citation follows...--- are erased).

I could not figure out why there is such difference. Would you mind sharing your thoughts? Please let me know in case my description above isn't clear.

I have realized that the problem is the notmuch implementation did not privode a org-msg-save-article-for-reply-notmuch function.

whudwl avatar May 04 '21 09:05 whudwl

@whudwl - thank you for intruding! That's precisely the issue. I really appreciate your help documenting what happens and why !

jonathanwilner avatar May 04 '21 15:05 jonathanwilner

@whudwl and @jonathanwilner, feel to volunteer and write this function. I do not use notmuch and I cannot write it.

jeremy-compostella avatar May 04 '21 17:05 jeremy-compostella

just quickly whipped up this piece which seems to work for me. I'm no expert on lisp so this could be totally far from an optimal solution.

(defvar org-msg-notmuch-saved-article-path "")

(defun org-msg-notmuch-save-article (handle)
  "recursively search through the result of (mm-dissect-buffer) and save the text/html part.
path of the saved file is saved in valiable org-msg-notmuch-saved-article-path
"
  (cond
   ((stringp (car handle)) (mapcar #'org-msg-notmuch-save-article (cdr handle)))
   ((bufferp (car handle))
    (if (string= (car (car (cdr handle))) "text/html")
        (progn
          (let ((file (make-temp-file "org-msg" nil ".html")))
            (mm-save-part-to-file handle file)
            (setq org-msg-notmuch-saved-article-path file)))))
   (t
    (mapcar #'org-msg-notmuch-save-article handle))))

(defun org-msg-save-article-for-reply-notmuch ()
  (let ((message-id  (save-excursion
                       (goto-char (point-min))
                       (re-search-forward "In-Reply-To: <\\(.*\\)>")
                       (match-string-no-properties 1))))
    (save-window-excursion
      (let ((notmuch-show-only-matching-messages t))
        (notmuch-show (format "id:%s" message-id)))
      (with-current-notmuch-show-message
       (org-msg-notmuch-save-article (mm-dissect-buffer)))
      (list org-msg-notmuch-saved-article-path))))

whudwl avatar May 05 '21 04:05 whudwl

This is interesting. While I was re-working you code I realized a few things:

  1. it does not save the images or re-bind them in the <img> tags
  2. it does create the header

Then I figured we could re-use the gnus engine.

Does the following work for you ? Note that I reworked some stuff too, so you may have to debug it a little bit.

(defun org-msg-save-article-for-reply-notmuch ()
  (let ((id (trim (org-msg-message-fetch-field "in-reply-to")))
	header parts)
    (cl-flet ((get-field (field)
	       (when-let ((value (org-msg-message-fetch-field field)))
		 (concat (capitalize field) ": " value))))
      (save-window-excursion
	(let ((notmuch-show-only-matching-messages t))
          (notmuch-show (format "id:%s" (substring id 1 -1))))
	(with-current-notmuch-show-message
	 (let ((fields (mapcar #'get-field
			       '("from" "subject" "to" "cc" "date"))))
	   (setf header (mapconcat 'identity (delq nil fields) "\n")))
	 (setf parts (mm-dissect-buffer)))
      (let* ((browse-url-browser-function #'ignore)
	     (save (cl-copy-list gnus-article-browse-html-temp-list)))
	(cl-letf (((symbol-function 'gnus-summary-show-article) #'ignore))
	  (save-window-excursion
	    (gnus-article-browse-html-parts parts header)))
	(let ((temp-files (cl-set-difference gnus-article-browse-html-temp-list save
					     :test 'string=)))
	  (setq gnus-article-browse-html-temp-list save)
	  temp-files))))))

It creates a code duplication in OrgMsg that if it that code work will have to be taken care of.

jeremy-compostella avatar May 06 '21 00:05 jeremy-compostella

This is interesting. While I was re-working you code I realized a few things:

  1. it does not save the images or re-bind them in the <img> tags
  2. it does create the header

Then I figured we could re-use the gnus engine.

Does the following work for you ? Note that I reworked some stuff too, so you may have to debug it a little bit.

(defun org-msg-save-article-for-reply-notmuch ()
  (let ((id (trim (org-msg-message-fetch-field "in-reply-to")))
	header parts)
    (cl-flet ((get-field (field)
	       (when-let ((value (org-msg-message-fetch-field field)))
		 (concat (capitalize field) ": " value))))
      (save-window-excursion
	(let ((notmuch-show-only-matching-messages t))
          (notmuch-show (format "id:%s" (substring id 1 -1))))
	(with-current-notmuch-show-message
	 (let ((fields (mapcar #'get-field
			       '("from" "subject" "to" "cc" "date"))))
	   (setf header (mapconcat 'identity (delq nil fields) "\n")))
	 (setf parts (mm-dissect-buffer)))
      (let* ((browse-url-browser-function #'ignore)
	     (save (cl-copy-list gnus-article-browse-html-temp-list)))
	(cl-letf (((symbol-function 'gnus-summary-show-article) #'ignore))
	  (save-window-excursion
	    (gnus-article-browse-html-parts parts header)))
	(let ((temp-files (cl-set-difference gnus-article-browse-html-temp-list save
					     :test 'string=)))
	  (setq gnus-article-browse-html-temp-list save)
	  temp-files))))))

It creates a code duplication in OrgMsg that if it that code work will have to be taken care of.

where does the (trim ...) function come from? My emacs couldn't find it.

whudwl avatar May 06 '21 00:05 whudwl

I don't know. I added it for safety but you can probably remove the call to this function.

On Wed, May 5, 2021, 5:57 PM David @.***> wrote:

This is interesting. While I was re-working you code I realized a few things:

  1. it does not save the images or re-bind them in the tags
  2. it does create the header

Then I figured we could re-use the gnus engine.

Does the following work for you ? Note that I reworked some stuff too, so you may have to debug it a little bit.

(defun org-msg-save-article-for-reply-notmuch () (let ((id (trim (org-msg-message-fetch-field "in-reply-to"))) header parts) (cl-flet ((get-field (field) (when-let ((value (org-msg-message-fetch-field field))) (concat (capitalize field) ": " value)))) (save-window-excursion (let ((notmuch-show-only-matching-messages t)) (notmuch-show (format "id:%s" (substring id 1 -1)))) (with-current-notmuch-show-message (let ((fields (mapcar #'get-field '("from" "subject" "to" "cc" "date")))) (setf header (mapconcat 'identity (delq nil fields) "\n"))) (setf parts (mm-dissect-buffer))) (let* ((browse-url-browser-function #'ignore) (save (cl-copy-list gnus-article-browse-html-temp-list))) (cl-letf (((symbol-function 'gnus-summary-show-article) #'ignore)) (save-window-excursion (gnus-article-browse-html-parts parts header))) (let ((temp-files (cl-set-difference gnus-article-browse-html-temp-list save :test 'string=))) (setq gnus-article-browse-html-temp-list save) temp-files))))))

It creates a code duplication in OrgMsg that if it that code work will have to be taken care of.

where does the (trim ...) function come from? My emacs couldn't find it.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/jeremy-compostella/org-msg/issues/73#issuecomment-833148516, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBHUAJRAFIR5F3AO7XTTMHSP7ANCNFSM4R7WSCZA .

jeremy-compostella avatar May 06 '21 02:05 jeremy-compostella

I don't know. I added it for safety but you can probably remove the call to this function. On Wed, May 5, 2021, 5:57 PM David @.**> wrote: This is interesting. While I was re-working you code I realized a few things: 1. it does not save the images or re-bind them in the tags 2. it does create the header Then I figured we could re-use the gnus engine. Does the following work for you ? Note that I reworked some stuff too, so you may have to debug it a little bit. (defun org-msg-save-article-for-reply-notmuch () (let ((id (trim (org-msg-message-fetch-field "in-reply-to"))) header parts) (cl-flet ((get-field (field) (when-let ((value (org-msg-message-fetch-field field))) (concat (capitalize field) ": " value)))) (save-window-excursion (let ((notmuch-show-only-matching-messages t)) (notmuch-show (format "id:%s" (substring id 1 -1)))) (with-current-notmuch-show-message (let ((fields (mapcar #'get-field '("from" "subject" "to" "cc" "date")))) (setf header (mapconcat 'identity (delq nil fields) "\n"))) (setf parts (mm-dissect-buffer))) (let ((browse-url-browser-function #'ignore) (save (cl-copy-list gnus-article-browse-html-temp-list))) (cl-letf (((symbol-function 'gnus-summary-show-article) #'ignore)) (save-window-excursion (gnus-article-browse-html-parts parts header))) (let ((temp-files (cl-set-difference gnus-article-browse-html-temp-list save :test 'string=))) (setq gnus-article-browse-html-temp-list save) temp-files)))))) It creates a code duplication in OrgMsg that if it that code work will have to be taken care of. where does the (trim ...) function come from? My emacs couldn't find it. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#73 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBHUAJRAFIR5F3AO7XTTMHSP7ANCNFSM4R7WSCZA .

I'm getting gnus-article-browse-html-parts: No buffer named *Article* error for most emails. any ideas what could be the issue?

whudwl avatar May 06 '21 02:05 whudwl

What do you mean by most emails? Does it work with some email? What is different in this emails?

jeremy-compostella avatar May 06 '21 03:05 jeremy-compostella

What do you mean by most emails? Does it work with some email? What is different in this emails!

Did a bit of more testing. Seems the error happens when there are inline images.

  1. Sent myself an empty email with an inline image. When replying to it, got the "No buffer named Article" error.
  2. Sent myself an empty email with an image attachment. Replying to it worked fine.

whudwl avatar May 06 '21 05:05 whudwl

What do you mean by most emails? Does it work with some email? What is different in this emails!

seems the error is emitted from line 2845 of gnus-art.el.

				(with-current-buffer gnus-article-buffer
				  gnus-article-mime-handles)

the value of the gnus-article-buffer variable is *Article* . Since I don't use gnus, I guess that's the default value.

whudwl avatar May 06 '21 05:05 whudwl

It seems gnus relies on the local variable gnus-article-mime-handles in the gnus-article-buffer to save cid contents. so how about this:

(defun org-msg-save-article-for-reply-notmuch ()
  (let ((id (string-trim (org-msg-message-fetch-field "in-reply-to")))
        header parts)
    (cl-flet ((get-field (field)
                         (when-let ((value (org-msg-message-fetch-field field)))
                           (concat (capitalize field) ": " value))))
      (save-window-excursion
        (let ((notmuch-show-only-matching-messages t))
          (notmuch-show (format "id:%s" (substring id 1 -1))))
        (with-current-notmuch-show-message
         (let ((fields (mapcar #'get-field
                               '("from" "subject" "to" "cc" "date"))))
           (setf header (mapconcat 'identity (delq nil fields) "\n")))
         (setf parts (mm-dissect-buffer)))
        (let* ((browse-url-browser-function #'ignore)
               (save (cl-copy-list gnus-article-browse-html-temp-list)))
          (cl-letf (((symbol-function 'gnus-summary-show-article) #'ignore))
            (save-window-excursion
              (setq-local gnus-article-mime-handles parts)
              (let ((gnus-article-buffer (current-buffer)))
                (gnus-article-browse-html-parts parts header))))
          (let ((temp-files (cl-set-difference gnus-article-browse-html-temp-list save
                                               :test 'string=)))
            (setq gnus-article-browse-html-temp-list save)
            temp-files))))))

whudwl avatar May 06 '21 06:05 whudwl

Even simpler:

(defun org-msg-save-article-for-reply-notmuch ()
  (let ((id (trim-string (org-msg-message-fetch-field "in-reply-to")))
	header parts)
    (cl-flet ((get-field (field)
	       (when-let ((value (org-msg-message-fetch-field field)))
		 (concat (capitalize field) ": " value))))
      (save-window-excursion
	(let ((notmuch-show-only-matching-messages t))
          (notmuch-show (format "id:%s" (substring id 1 -1))))
	(with-current-notmuch-show-message
	 (let ((fields (mapcar #'get-field
			       '("from" "subject" "to" "cc" "date"))))
	   (setf header (mapconcat 'identity (delq nil fields) "\n")))
	 (setf parts (mm-dissect-buffer))))
      (with-temp-buffer
	(let ((gnus-article-buffer (current-buffer))
	      (gnus-article-mime-handles parts))
	  (let* ((browse-url-browser-function #'ignore)
		 (save (cl-copy-list gnus-article-browse-html-temp-list)))
	    (cl-letf (((symbol-function 'gnus-summary-show-article) #'ignore))
	      (save-window-excursion
		(gnus-article-browse-html-parts parts header)))
	    (let ((temp-files (cl-set-difference gnus-article-browse-html-temp-list save
						 :test 'string=)))
	      (setq gnus-article-browse-html-temp-list save)
	      temp-files)))))))

jeremy-compostella avatar May 06 '21 16:05 jeremy-compostella

I pushed the improved changes to the experimental branch. On this branch I also implemented org-msg-article-htmlp-notmuch. I don't use notmuch so I can't really test. Could you test and debug the experimental branch ? I tried to unitary test what I can but notmuch specific function are out of reach.

jeremy-compostella avatar May 06 '21 18:05 jeremy-compostella

@jeremy-compostella Thank you so much for that. I did a bit of testing of the experimental branch, made some minor fixes. Now it pretty much works perfectly for me. Have created pull request.

whudwl avatar May 06 '21 22:05 whudwl

I've also tested the experimental branch and it's working for me as well using notmuch! Thanks @whudwl + @jeremy-compostella !

jonathanwilner avatar May 06 '21 22:05 jonathanwilner

I've also tested the experimental branch and it's working for me as well using notmuch! Thanks @whudwl + @jeremy-compostella !

Emm. Strange. It worked for you as it is? I had to modify it a bit for it to work.

whudwl avatar May 06 '21 22:05 whudwl

Yes - each one of these has worked for me out of the box - except for "trim" - which I couldn't find either.

The current experimental branch has a weird "select all" effect where all of the previous message is selected for yank that is going on when the reply is first composed, but it's not holding me back too much.

jonathanwilner avatar May 06 '21 22:05 jonathanwilner

I saw that “select all” effect too. I think it’s the result of an error. Lets see if it’s fixed once jemery pulls my change. On 7 May 2021, 8:56 AM +1000, jonathanwilner @.***>, wrote:

Yes - each one of these has worked for me out of the box - except for "trim" - which I couldn't find either. The current experimental branch has a weird "select all" effect where all of the previous message is selected for yank that is going on when the reply is first composed, but it's not holding me back too much. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

whudwl avatar May 06 '21 23:05 whudwl

@whudwl I integrated all your changes. Let me know if it works better. BTW, I removed the call to string-trim because the message-fetch-field function already does it.

jeremy-compostella avatar May 06 '21 23:05 jeremy-compostella

@jonathanwilner and @whudwl Is the "select all" effect gone now ?

jeremy-compostella avatar May 07 '21 16:05 jeremy-compostella

Yes. Now it works perfectly.

whudwl avatar May 07 '21 17:05 whudwl

Yes! Thank you !!!

jonathanwilner avatar May 07 '21 18:05 jonathanwilner

@jeremy-compostella , could you perhaps add a line to kill the notmuch buffer after using it? (notmuch-show) Returns the buffer containing the messages. I was testing and replying to the same message again and again, and I end up with a dozen buffers showing the same message.

whudwl avatar May 08 '21 05:05 whudwl

I suggest that you submit a patch.

On Fri, May 7, 2021, 10:25 PM David @.***> wrote:

@jeremy-compostella https://github.com/jeremy-compostella , could you perhaps add a line to kill the notmuch buffer after (notmuch-show) call? (notmuch-show) Returns the buffer containing the messages. I was testing and replying to the same message again and again, and I end up with a dozen buffers showing the same message.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jeremy-compostella/org-msg/issues/73#issuecomment-835110059, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMACBATJSZDQUULYL7WE33TMTDMNANCNFSM4R7WSCZA .

jeremy-compostella avatar May 08 '21 05:05 jeremy-compostella

@whudwl Let me know if 9be11351 works.

jeremy-compostella avatar May 10 '21 17:05 jeremy-compostella