nbdev
nbdev copied to clipboard
`nbdev_test` breaks on Apple M1 due to `BrokenAccessPool`
nbdev is working great for me on Linux VMs, but every time I try to run nbdev_test
on my Macbook Pro M1 I run into the error below. nbdev_export
is running fine.
Traceback for nbdev_test
call:
Traceback (most recent call last):
File "/opt/anaconda3/envs/ENV/bin/nbdev_prepare", line 8, in <module>
sys.exit(prepare())
File "/opt/anaconda3/envs/ENV/lib/python3.7/site-packages/fastcore/script.py", line 119, in _f
return tfunc(**merge(args, args_from_prog(func, xtra)))
File "/opt/anaconda3/envs/ENV/lib/python3.7/site-packages/nbdev/quarto.py", line 238, in prepare
nbdev.test.nbdev_test.__wrapped__()
File "/opt/anaconda3/envs/ENV/lib/python3.7/site-packages/nbdev/test.py", line 90, in nbdev_test
basepath=get_config().config_path, pause=pause, do_print=do_print, **kw)
File "/opt/anaconda3/envs/ENV/lib/python3.7/site-packages/fastcore/parallel.py", line 117, in parallel
return L(r)
File "/opt/anaconda3/envs/ENV/lib/python3.7/site-packages/fastcore/foundation.py", line 98, in __call__
return super().__call__(x, *args, **kwargs)
File "/opt/anaconda3/envs/ENV/lib/python3.7/site-packages/fastcore/foundation.py", line 106, in __init__
items = listify(items, *rest, use_list=use_list, match=match)
File "/opt/anaconda3/envs/ENV/lib/python3.7/site-packages/fastcore/basics.py", line 66, in listify
elif is_iter(o): res = list(o)
File "/opt/anaconda3/envs/ENV/lib/python3.7/concurrent/futures/process.py", line 483, in _chain_from_iterable_of_lists
for element in iterable:
File "/opt/anaconda3/envs/ENV/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
yield fs.pop().result()
File "/opt/anaconda3/envs/ENV/lib/python3.7/concurrent/futures/_base.py", line 435, in result
return self.__get_result()
File "/opt/anaconda3/envs/ENV/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
The same error occurs in a Python 3.9 environment. Am I right to think this is an issue specific to Apple M1s hardware or could it be a more general problem? I'm using concurrent.futures.ThreadPoolExecutor
functionality in my code so it could be related to that.
I found that I have to export the following env variable on M1 Mac
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
cc: @jph00
I bet this is either related or a dupe of: https://github.com/fastai/nbdev/issues/731 https://github.com/fastai/nbdev/issues/693 https://github.com/fastai/nbdev/issues/673