pythontex
pythontex copied to clipboard
Support for platforms where multiprocessing does not work
Hi,
Currently trying to run pythontex on android gives an error due to multiprocessing not working. There's been plenty of discussion on how to deal with this in other python projects, see for example https://github.com/pypa/pip/issues/8161.
With this PR we check if multiprocessing is supported and else set multiprocessing=False. When later creating list of tasks to run we then choose to either run pool.apply_async(function), or to just eval(function).
Error message when currently running pythontex
This is PythonTeX 0.17
Traceback (most recent call last):
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/synchronize.py", line 28, in <module>
from _multiprocessing import SemLock, sem_unlink
ImportError: cannot import name 'SemLock' from '_multiprocessing' (/data/data/com.termux/files/usr/lib/python3.8/lib-dynload/_multiprocessing.cpython-38.so)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/data/com.termux/files/usr/bin/texlive/pythontex", line 62, in <module>
pythontex.main()
File "/data/data/com.termux/files/usr/share/texlive/texmf-dist/scripts/pythontex/pythontex3.py", line 2811, in main
do_multiprocessing(data, temp_data, old_data, engine_dict)
File "/data/data/com.termux/files/usr/share/texlive/texmf-dist/scripts/pythontex/pythontex3.py", line 1269, in do_multiprocessing
pool = multiprocessing.Pool(jobs)
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/pool.py", line 191, in __init__
self._setup_queues()
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/pool.py", line 343, in _setup_queues
self._inqueue = self._ctx.SimpleQueue()
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/context.py", line 113, in SimpleQueue
return SimpleQueue(ctx=self.get_context())
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/queues.py", line 336, in __init__
self._rlock = ctx.Lock()
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/context.py", line 67, in Lock
from .synchronize import Lock
File "/data/data/com.termux/files/usr/lib/python3.8/multiprocessing/synchronize.py", line 30, in <module>
raise ImportError("This platform lacks a functioning sem_open" +
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.
I have not updated pythontex2.py, mainly due to that 99 % of all users running into this problem will be using the terminal emulator termux (as I am) where python3 is already the default python version.
Let me know if you are not happy with the code format, associated comments or if I should fix something else in the commits!