tikz typesets missing characters, and \tracinglostchars

[kberry]

I know you know this, but tikz typesets characters from nullfont with some regularity. For example:

\documentclass{article}
\usepackage{tikzlings}
\begin{document}
\tikz\bear;
\end{document}

yields

Missing character: There is no ; in font nullfont!

Ulrike assues me that the tikz being generated by \bear is perfectly valid :); I've also seen missing characters in plenty of non-tikzling cases.

Since missing characters are usually a critical problem, indicating incorrect output, it is painful that these particular missing characters have to be ignored. It would be nice if they were not incorrectly typeset in the first place.

With the advent of \tracinglostchars=2 by default for xe/lualatex, I expect the error will get noticed by many more people. And it will be a stopper for people setting \tracinglostchars=3. If the real fix is not plausibly going to happen, maybe tikz should decrease \tracinglostchars to 1 if it is 2 or 3 ... --best, karl.

[muzimuzhi]

If the real fix is not plausibly going to happen, maybe tikz should decrease \tracinglostchars to 1 if it is 2 or 3 ... --best, karl.

Already done in #1018 (a maximum of 2 for \tracinglostchars is added), though not released yet.

[hmenke]

Every lost character that is not obviously a user error should be reported as a bug.

The present one, however, is a really tricky. It can be reproduced with a much simpler MWE:

\documentclass{article}
\usepackage{tikz}
\begin{document}
\tracinglostchars=3
\errorcontextlines=\maxdimen
\tikz\begin{scope}\end{scope};
\end{document}

Under the hood TikZ rewrites

\tikz\begin{scope}\end{scope};

to something equivalent to

\begin{tikzpicture}\begin{scope}\end{scope};\end{tikzpicture}

but in this case the semicolon is not necessary at this point and appears as a lost character but TikZ can't know that. So we can't just strip the semicolon and hope for the best, because then any path (e.g. \tikz\draw (0,0) -- (1,1);) would be missing its semicolon.

To solve this we need to perform surgery on the parser to not only look for ; but also \endtikzpicture. Then we can drop the semicolon in the rewritten picture.

[kberry]

Isn't parsing separate from typesetting? Or is that not the case with tikz?

If current font is nullfont, never typeset anything.

[muzimuzhi]

\tikz is documented to accept path commands, but what syntax role the scope env, or anything that doesn't need a semicolon to stop parsing, plays is unclear.

[muzimuzhi]

@kberry Parsing only starts with path commands (\path, \draw, ...), and it ends when the commands finish with a semicolon. In other words, both \tikz and \begin{tikzpicture} are not triggers. Hence if the content, or the remaining content after (previously finished) path commands, of a tikzpicuter env doesn't start with any path commands, it typesets. For example \tikz {text} will typeset text in nullfont, and

\begin{tikzpicture}\begin{scope}\end{scope};\end{tikzpicture}

will typeset the extra ; in nullfont as \nullfont is already called in \tikzpicture/\pgfpicture.