Option to not show preview until mathjax is finished rendering.

[michaelerule]

First: beautiful work. I've tried many markdown editors on Linux and this is by far my favorite so far. Simple, responsive, and stable.

I don't know how hard this could be to add, but I've noticed that the live preview jumps around considerably when rendering if mathjax is enabled. I find this quite distracting.

It seems like the editor is aware of the progress of math rendering, since displays a status message that ends with "Typesetting math: 100%"

I wonder, how hard would it be to add an option to delay refresh until math rendering is done, to reduce jitter? Alternatively, there could be an option to reload the live preview only upon some keystroke, so the user has control over when all this happens?

Thanks again! -m

[michaelerule]

I took a brief look at the source code. I wasn't fully able to figure things out, but this is what I was able to get:

• The app is mostly written in Python, calling out to the QT library for the GUI elements
• It uses some sort of HTML rendering / markdown rendering library as well
• I couldn't find the point in the code where the live rendering is triggered. If I can though, I'd like to try hacking it to re-render only on a custom keystroke.
• I'm not sure how the rendering progress status messages are generated. They could be generated from Python, or perhaps are an automatic part of the markdown rendering library? Some stuff like MathJax seems to happen in Javascript, so there may be some way to arrange things so that rendering is paused until MathJax is finished. It might even be possible to patch/tweak the mathjax extension to achieve this?

[mitya57]

Hi!

ReText just inserts the <script> element that loads MathJax into the page, everything else happens on JS side and is handled by the web engine.

MathJax does have an event that is fired when the content finished loading: http://docs.mathjax.org/en/latest/web/configuration.html#performing-actions-after-typesetting

However, it would be quite difficult to connect to that event. The only way I can imagine is using WebSockets to send some message from client code to the app (and the app will need to use Qt WebSockets API).

But things get much worse because:

• ReText supports two different web engines (WebKit and WebEngine).
• ReText (or the underlying PyMarkups library, to be more precise) supports both MathJax v2 and v3, and v3 was a complete rewrite with different API. This is the state in current master branch, the stable release supports only v2.
• Code that would implement such integration would have to deal with integration between ReText and PyMarkups, and maybe also with integration between two ReText processes (rendering happens in a separate process).

That said, pull requests are welcome, but it won't be easy.

[michaelerule]

Interesting;

Any hope of modifying the mathjax plugin or the generated HTML along these lines: https://superuser.com/questions/680425/mathjax-hide-page-until-all-math-formulas-are-typesetted ?

[mitya57]

Any hope of modifying the mathjax plugin or the generated HTML along these lines […]?

This is not what you asked for — it does not delay the page refreshing. It just hides all the content until the math is typeset.

You can try to add something like this to your document to get the idea (works with MathJax 2.7):

<script>
(function() {
  document.body.style.visibility = 'hidden';
})();
MathJax.Hub.Queue(function() {
  document.body.style.visibility = 'visible';
});
</script>

Or you can look at this example: https://www.tuhh.de/MathJax/test/sample-all-at-once.html.

In my opinion, it is still a flicker, maybe even worse than the default behavior.

[michaelerule]

Nice! I had no idea one could just throw script tags into markdown source .

As someone with ADD, to me the blank white screen is a slight improvement. If I could tweak ReText to only trigger a preview re-render, say, on save (or another keystroke), it might be ok: I could do side-by-side updates to check math rendering, but avoid noise in my peripheral vision when I am doing other tasks, e.g. writing text.

It would be less bad if mathjax didn't change the vertical spacing, causing the text and scroll bar to jump all over the screen for a few seconds.

[mitya57]

Nice! I had no idea one could just throw script tags into markdown source .

Yes, you can!

If I could tweak ReText to only trigger a preview re-render, say, on save (or another keystroke), it might be ok: I could do side-by-side updates to check math rendering, but avoid noise in my peripheral vision when I am doing other tasks, e.g. writing text.

Can I suggest you to fork ReText and make the modifications locally?

You can try to comment out this line to disable preview update on editing: https://github.com/retext-project/retext/blob/db04a36cd63a66c988d7f551a5cc83cbbb34eed1/ReText/tab.py#L87

Or maybe just increase the timeout here — it will not disable preview update but make it less frequent: https://github.com/retext-project/retext/blob/db04a36cd63a66c988d7f551a5cc83cbbb34eed1/ReText/tab.py#L283

If you want to refresh on save, you can add a self.triggerPreviewUpdate() to saveTextToFile method in the same file.

It would be less bad if mathjax didn't change the vertical spacing, causing the text and scroll bar to jump all over the screen for a few seconds.

I have several ideas that can make it a bit better. To try them, you will need to patch code in PyMarkups, not in ReText. Clone the repository, run python3 setup.py develop there, then you can make changes and ReText will automatically use them.

The modifications will be in this block: https://github.com/retext-project/pymarkups/blob/0eef84e7910539ec7a554029934fbaa2808e4a5e/markups/markdown.py#L21-L29

Idea 1: try appending "fast-preview.js" to the list of extensions. It will tell mathjax to show a “preview” — a simplified version of your formulae.

Idea 2: try replacing output/HTML-CSS with output/CommonHTML. This should make rendering the final version of math a bit faster.

Please let me know if any of this makes the situation better — then maybe I will make it configurable more easily.

[michaelerule]

Thanks!

• I made the suggested changes in the ReText source; Thanks for pointing me in the right direction.
• fast-preview.js doesn't help: it just jumps around while the fast preview renders, then jumps around some more as the rendered math doesn't have the same height as the fast preview
• I didn't notice any change for output/HTML-CSS -> output/CommonHTML, except perhaps that rendering was a bit slower.

It sounds like this issue will be hard to fix well: MathJax needs to run as javascript, which can only happen after the HTML is prepared. Blanking the screen while rendering is not ideal. One work-around might be to find a way to prepare the HTML such that MathJax does not alter the vertical height of math blocks before/after rendering. But, this would require changing the markdown -> HTML conversion libraries (or post-processing their outputs)?

Or... doing the first thing, and figuring out how to double-buffer the preview window, and reliably catch the various signals from Mathjax in all its incarnations. My suspicion is that this would mostly introduce bugs/instability/unreliability, e.g. never showing the preview if a signal doesn't make it across a socket for some reason.

[mitya57]

One work-around might be to find a way to prepare the HTML such that MathJax does not alter the vertical height of math blocks before/after rendering. But, this would require changing the markdown -> HTML conversion libraries (or post-processing their outputs)?

But how can we know the height of math formulae before they are actually rendered?

There is one more thing, but I doubt it will help: try putting this line on top of your document:

<!-- Required extensions: mdx_math(add_preview=True) -->

It will show the source code of formula until math is rendered. But the source code will have different height still…

My suspicion is that this would mostly introduce bugs/instability/unreliability, e.g. never showing the preview if a signal doesn't make it across a socket for some reason.

Exactly. It is hard to get such things right.

[michaelerule]

The only way I can think to know the equation height is to grab the height after mathjax is finished rendering, and store it in a cache somewhere. Then any equations that aren't currently changing can be re-rendered without the vertical height jumping around. Any equations containing \ref or \eqref might change their height if the tags pointed to by those references are updated, so the equation source code doesn't even uniquely determine the height. This is heading towards a solution that breaks the document down into blocks that are rendered independently then cached. Too complex.

There might be some hack of creating an HTML document that contains, initially, the mathjax-processed HTML document from before, then uses the javascript callback mentioned earlier to manually flip to the updated version (assuming mathjax can be convinced to render in the background). Also terrible.

I wonder how vscode does it. I guess, probably a different renderer entirely -- or at least finer integration of the window system with the markdown renderer, so it doesn't produce (or doesn't show) any unprocessed HTML. (vscode still can't render equation numbers though) meh.

[mitya57]

Right, it's very complicated.

Maybe using an alternative library, such as KaTeX, will be better?

Can you try to replace these lines: https://github.com/retext-project/pymarkups/blob/0eef84e7910539ec7a554029934fbaa2808e4a5e/markups/markdown.py#L245-L249

With the following?

return """
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/katex/dist/katex.min.css" crossorigin="anonymous">
<script src="https://cdn.jsdelivr.net/npm/katex/dist/katex.min.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/katex/dist/contrib/mathtex-script-type.min.js" defer></script>
"""

[michaelerule]

Ah!

KaTeX

• Is faster, so the flicker doesn't last as long
• Seems to get the vertical hight correct, so things don't jump as much
• Seems to ignore the "don't show until rendering is finished" javascript from earlier?
• Can't seem to handle the \begin{equation} tag, so I'm not sure how to get numbered equations.

Generally: improved rendering, reduced flicker, but lost some features I had been using. This renderer is prefereable, I think, if I can get back equation numbers/references.

Edit: Ah! KaTeX supports manual equation labelling via the \tag{} command; Unclear to me if autonumbering is supported. But overall, much improved performance.

[mitya57]

Seems to ignore the "don't show until rendering is finished" javascript from earlier?

Yes, that javascript was using MathJax API, while KaTeX is a completely different code base.

Can't seem to handle the \begin{equation} tag, so I'm not sure how to get numbered equations.

Actually support for that was added in KaTeX/KaTeX#2369, but there is no release that includes that yet. So you can either build KaTeX yourself or wait until the next release (or use \tag{} in the meantime, as you noted).