[WIP] TeXpresso integration

[tsung-ju]

TeXpresso (by @let-def) is a program that provides a "live rendering" experience when editing LaTeX documents. This PR aims to integrate TeXpresso into VimTeX as a compiler.

Unlike other latex compilers, TeXpresso combines a compiler based on tectonic and a pdf viewer based on MuPDF, and communicates file changes, quickfix items, and synctex through stdio. In this PR, these are implemented by opening a continuous compiler and changing the input mode to 'pipe' in autoload/vimtex/compiler/_template.vim.

Most main features of TeXpresso including live editing, synctex, quickfix, theme color syncing are working. However, the code still have some rough edges (hence marked as draft). Please feel free to try the code out. Any suggestion would be really appreciated.

[lervag]

First: I'm very happy to see such a contribution! And I look forward to reviewing the code. But I'm on vacation right now, and I can't promise that I'll be able to review this immediately. I'll look at it as soon as I get the time!

TeXpresso (by @let-def) is a program that provides a "live rendering" experience when editing LaTeX documents. This PR aims to integrate TeXpresso into VimTeX as a compiler.

I was not aware of TeXpresso. It looks very interesting. It would be nice to hear more about how it works - how is it possible to be "so fast" wrt. compile times? Does it work well even for large and complex documents?

[let-def]

Hi @tsung-ju & @lervag, I am glad to see this contribution. I don't have much free time these days, so I will only follow this from a distance.

I was not aware of TeXpresso. It looks very interesting. It would be nice to hear more about how it works - how is it possible to be "so fast" wrt. compile times?

I can explain a bit how TeXpresso works. The two main pieces are the engine, a fork of XeTeX, and the driver, which plays two roles: feeding the engine and rendering the output.

Let's look at the engine first. It's a fork of XeTeX, currently the one found in the Tectonic distribution, but i would like to synchronize with TeXlive in the future. The main purpose of the fork is to redirect the I/O, so that the data can come from a custom source, e.g. an editor mode, and not necessarily from the file system. This is also used to keep track of the contents observed by the TeX process, in order to recompute things only when they change.

The driver use this to (1) feed the engine with data coming either from the file system, a cache of the TeX packages (this directly comes from Tectonic at the moment) or the buffers of the editor, and (2) take snapshots from time to time by forking the engine (fork as in unix's fork(2)). When contents change in the editor, the TeX process is resumed from the fork closest to the change. On average, only the visible page has to be recomputed.

The rendering pipeline is entirely custom and is built on top of the DVI/XDV format. Mupdf is not really used as a PDF renderer but rather as a framework for managing fonts and drawing to a canvas. The PDF renderer is used only to implement \includegraphics with pdf files.

Does it work well even for large and complex documents?

I think it works well with large and complex documents, as long as they are XeTeX compatible. I have tested it with books full of graphics (e.g. https://github.com/fabiensanglard/gebbdoom) and large math documents (close to a thousand pages). There are some limitations though. First, TeXpresso does only a single pass: to have a table of contents or a bibliography, one will need to run LaTeX or BiBTeX passes separately. TeXpresso can use the aux files produced by an external build system. It checks for update to these files when receiving SIGUSR1, so appending killall -SIGUSR1 texpresso to a build script is usually sufficient for a seamless integration. Second, the project is quite experimental. I did the minimum to provide a decent experience for my use cases, but it is far from being rock solid. I experience a crash every once in a while, but it can vary a lot depending on the features used (I debugged the things I use the most, the rest is alpha quality :)). We also found some bugs in neovim, e.g. https://github.com/neovim/neovim/issues/24931, so the mode can desynchronize from time to time. :e! is sufficient to force a resynchronization, so I did not bother much with that.

[lervag]

Hi @tsung-ju & @lervag, I am glad to see this contribution. I don't have much free time these days, so I will only follow this from a distance.

👍

I was not aware of TeXpresso. It looks very interesting. It would be nice to hear more about how it works - how is it possible to be "so fast" wrt. compile times?

I can explain a bit how TeXpresso works. …

Thanks for the info! Sounds like this may be a very interesting idea and a useful addition to VimTeX!

[lervag]

I've looked at the code and I think in general it seems very good. Thanks for following my conventions!

I've not tested it myself yet. I look forward to testing it.

One major thing that is missing for now is documentation. I think a nice starting point is to add something similar to the docs for _latexrun.

[lervag]

Are you still interested in getting this contribution merged?

[tsung-ju]

Hi, Sorry, but I no longer have time to work on this. I'm closing it for now.

[lervag]

Sounds good. Feel free to reopen if you should ever find time!