[mu4e bug] starting mu4e messes with shell-command over tramp

[mbunkus]

Describe the bug

This is very weird and might totally not be a bug in mu4e itself.

Whenever you run shell-command and it outputs more than one but fewer than a number of lines determined by max-mini-window-height, usually those lines are displayed in a temporarily expanded minibuffer. The minibuffer remains expanded until you press the next key when it returns to its original size & no content being displayed. This works fine over tramp, too, and is basic functionality of Emacs.

However, starting either with some recent Emacs update or some recent mu4e update this behavior changes as soon as I start mu4e with the command mu4e (no matter if called interactively or from elisp). As soon as mu4e has been started once, running the same shell command will still cause the command to run, but the minibuffer will no longer expand & display the text as it did before. What seems to happen is that the minibuffer is expanded, the command's output is displayed, and the minibuffer contracts immediately as if I'd pressed a key — which I haven't.

This only happens when running shell-command over tramp, e.g. when executing it from a file opened via tramp. It does not happen if run locally. It keeps happening until I restart Emacs without running mu4e.

Simply loading mu4e does not trigger the behavior change — only starting mu4e itself.

Note that the *Shell Command Ouptut* buffer does contain the generated output, meaning the command is actually run — it's just a display issue with the minibuffer.

How to Reproduce

Prepare a directory on a machine you have ssh access to (can be localhost). Place two files in it.

Start Emacs without the user files but with mu4e available. For me this means a simple emacs -Q, adding the path to mu4e's packaged files to the load-path. Add your local development copy's path as you see fit. Load one of the two files from the prepared directory.

What I'm actually using is (split up into multiple lines for legibility):

emacs -Q -nw --eval '
  (progn
    (add-to-list '\''load-path "/usr/share/emacs/site-lisp/mu4e")
    (require '\''mu4e)
    (find-file "/ssh:user@server:/path/to/prepared/directory/file1.txt"))'

First execute M-x shell-command (or M-!) & enter ls -l. What should happen is that your minibuffer expands slightly & shows the the three lines of output from ls. This is how it should look.

Now run M-x mu4e. It doesn't matter if it does anything at all or not.

Last re-run M-! ls -l and observe the wrong behavior described above: minibuffer shortly flickering (might not even be noticeable), not remaining expanded, not showing the output.

Caveat

I'm highly confused by this problem. I could totally understand if this isn't a bug in mu4e after all. What I'd really like is some kind of exchange of ideas how to debug this properly. Maybe something about my description will even ring a bell for you, pointing to possible culprits. Thanks!

Environment

• Arch Linux
• Emacs 29.1 from Arch's packages (the emacs-nativecomp package)
• mu & mu4e 1.10.5 from Arch's package
• with emacs -Q, max-mini-window-height is 0.25 & resize-mini-windowsisgrow-only`

Checklist

• [x] you are running either an 1.8.x/1.10.x release or master (otherwise please upgrade)
• [x] you can reproduce the problem without 3rd party extensions (including Doom/Evil, various extensions etc.)
• [x] you have read all of the above

[djcb]

Hmmm... a lot of moving parts here. Can reproduce without ssh/tramp? And what about an older mu4e version, older emacs? When did this behavior change?

[mbunkus]

I'm pretty sure I've never had this happen locally (not over tramp). But I'll give it a more thorough try again this weekend. I'll also try to downgrade Emacs & mu4e (separately) in order to determine when things changed.

[mbunkus]

Unfortunately I haven't been able to find a combination of Emacs & mu4e for which this works. Here's what I tested:

Emacs	mu4e	works?
29.1-4 nativecomp	1.10.5-1	✗
29.1-4 normal	1.10.5-1	✗
28.2-2 normal	1.10.5-1	✗
29.1-4 nativecomp	1.11.15 (0b62640962a5adc60f85e88d5b7512a333e577d6)	✗
29.1.4 nativecomp	v1.10.0	✗

Emacs packages before that 28.2 don't run anymore when installed in my current Arch. For mu4e compiling v1.8.14 & anything older fails due to something that looks like changes in gcc/g++ & its include files (types from stdint.h not being known).

None of those versions work, unfortunately.

So far I also haven't been able to reproduce the issue without tramp, meaning that anything local does work, no matter the version.

Any ideas how I might go about debugging further?

[mbunkus]

Some more digging: it starts to happen as soon as (mu4e-modeline-mode) is called within (mu4e) and stops happening if I call (mu4e-modeline-mode) in order to toggle it off again. It starts to happen again as soon as I turn it on with yet another call to (mu4e-modeline-mode).

Is there perhaps anything related to modeline updates that always gets called as soon as a shell-command is done & that itself runs another shell-command or anything else that uses the same minibuffer expansion mechanism?

[djcb]

Ah, some good detective work!

Can you reproduce with graphical emacs? And perhaps you can try with setting mu4e-use-fancy-chars to nil (if not already); perhaps it's some alignment thing.

[mbunkus]

Can you reproduce with graphical emacs?

Yes, I can (running Emacs inside an Xvnc session on a server here at home & can reproduce it there, but I also run a regular Linux desktop at work where I do run Emamcs in graphical mode & where it also happens).

And perhaps you can try with setting mu4e-use-fancy-chars to nil (if not already);

Doesn't help, unfortunately. Here's my latest startup command that triggers the issue:

emacs -Q --eval '
  (progn
    (add-to-list '\''load-path "/usr/share/emacs/site-lisp/mu4e")
    (require '\''mu4e)
    (setq mu4e-use-fancy-chars nil)
    (mu4e-modeline-mode)
    (find-file "/ssh:mosu@sweet-chili:/home/mosu/tmp/2f/a.txt")
    (shell-command "ls -l"))'

What I said earlier about toggling mu4e-modeline-mode on & off still holds in graphical mode: if disabled, everything's OK; if enabled, the issue is there.

[djcb]

I really have no idea then, sorry :-( It seems like an emacs bug, but it's a bit hard to isolate the precise problem for a bug-report. Perhaps turn of the mu4e-modeline then, if it annoys you enough?

[mbunkus]

Yeah, I've chosen to disable it for the time being as I don't have the time (or rather, the motivation) to dig into this very unfamiliar territory further. Thanks for your thoughts on it so far!

[djcb]

I tried this today, with 1.12.x and emacs 29.4, am not able to reproduce... It also seems a bit beyond what mu4e could cause / should be able to cause. It's also weird how the remote shell commands would make a difference here!

So, I'm going to close this as "CANTFIX" or something like that :) If you can see this still with 1.12.x and (perhaps) the next Emacs version, and/or perhaps are able to tweak mu4e-modeline.el to show the problem or avoid it, perhaps we can reopen, but right now this does not seem actionable, sorry!

[mbunkus]

That's totally fair. Thanks for working with me on it!