gpoore
After TeX capacity exceeded PythonTeX fails
When I compile a document which gets a (too) huge code back from Python pdflatex obviously fails with a message like: "! TeX capacity exceeded, sorry [input stack size=5000]."
When I now try to change the code so it will returns a small enough code from Python it fails again although it shouldn't.
The solution is to delete the "pythontex-files-treepy" folder. Would be nice to first check this folder or resp. the log file if last run exceeded capacity and if so to delete the folder "pythontex-files-treepy".
Greets
Just to make sure I'm clear on when this is happening, were you trying to bring back a huge chunk of text using \py, and then after the error had to switch to \pyc with print() or pycode with print()? Or were you sending back a line of text that was too long, and needed line breaks?
Checking the log is a good idea, assuming that the "TeX capacity exceeded" message can reliably be traced back to PythonTeX rather than other sources. Another possible solution is for PythonTeX always to check output for lines that are too long, or chunks of text that are too large to be stored in memory. I will try to incorporate some sort of check in the future.
I can confirm the described behavior. I had (by accident) produced way too much output in a pycode via print(). To recover from the capacity error I clean-out the temp files.
I'm not 100% sure. You can recreate the problem by using my code: http://tex.stackexchange.com/questions/131689/how-to-automatically-draw-tree-diagram-of-prime-factorization-with-latex/230348#230348 for instance if you run \PrimeTree{340282366920938463463374607431768211456} PS: I know that such a tree wouldn't be possible, just as an example for fixing this issue.
This problem is a special case of any scenario where pythontex tries to include illegal LaTeX into the document.
For instance print(r"\foo") will most likely get latex complaining that \foo is undefined. One solution is then to clear the pythontex directory, fix the bug and then re-compile. This is annoying, so if you can think of a better workaround, that would be great.
On 27 Feb 2015, at 15:25, unidesign [email protected] wrote:
I'm not 100% sure. You can recreate the problem by using my code: http://tex.stackexchange.com/questions/131689/how-to-automatically-draw-tree-diagram-of-prime-factorization-with-latex/230348#230348 for instance if you run \PrimeTree{340282366920938463463374607431768211456} PS: I know that such a tree wouldn't be possible, just as an example for fixing this issue.
— Reply to this email directly or view it on GitHub.
I'm pretty sure you're wrong with your assumption. The code works for all numbers which do not exceed the memory. And those numbers using the same code. Also all strings are properly escaped.
Yes, sorry for mixing in here. My point is that the bug you have reported is more general, any illegal pythontex output is not treated well by latex, although the out of memory is simpler/realistic to fix Python side.
To re-iterate: I do believe what you are reporting is a bug. But the bug is larger than what you have reported.
On 27 Feb 2015, at 16:03, unidesign [email protected] wrote:
I'm pretty sure you're wrong with your assumption. The code works for all numbers which do not exceed the memory. And those numbers using the same code. Also all strings are properly escaped.
— Reply to this email directly or view it on GitHub.
Okay thanks for pointing this out, now I understand the conherence. As you wrote it would be than more complex to fix the bug unfortunately.
The linked TeX.SE code is creating a single, long string. So the problem there is that the line is too long. It shouldn't be hard to detect that sort of thing on the Python side, and either raise an error, or possibly insert line breaks automatically and raise a warning.
In terms of the more general issue raised by @obtitus, if Python generates invalid LaTeX code (not just a line that is too long), there isn't a way to catch that on the Python side, because the only general way to check for invalid LaTeX is to run LaTeX. When Python generates invalid LaTeX, it can prevent a successful compile, which can prevent debugged code from being extracted and passed to PythonTeX. Deleting the pythontex directory eliminates the invalid code, allowing a successful compile.
My existing solution for this is the debug package option. This disables the inclusion of all Python-generated content, so that the document can be compiled and then PythonTeX can be run with fixed code. Using debug, you can fix bugs without having to delete the pythontex directory. Deleting the directory is inefficient if you are using multiple sessions, since typically only one will be causing a problem.
If anyone has suggestions for improving debug or dealing with the general case, I'd be happy to hear them. In the next major release, I will probably allow debug or an equivalent on a per-session or maybe even per-command/environment basis. And I'll plan on adding a check for over-long lines on the Python side.
Thanks that you analyzed the problem, good to hear that you say it could be possible to determine it on the Python side.