Notebook launched with w_jupyter throws pickling error when calling w_main()

[jdrusso]

When launching a Jupyter notebook for analysis with w_jupyter using a Python 3 kernel, the following cell

%matplotlib inline
from matplotlib import pyplot as plt
import numpy as np
import w_ipa
w = w_ipa.WIPI()
# At startup, it will load or run the analysis schemes specified in the configuration file (typically west.cfg)
w.main()

fails on the w.main() call with

--------------------------------------------------------------------
TypeError                          Traceback (most recent call last)
<ipython-input-1-e8838ebefc68> in <module>
      5 w = w_ipa.WIPI()
      6 # At startup, it will load or run the analysis schemes specified in the configuration file (typically west.cfg)
----> 7 w.main()
      8 w.interface = 'matplotlib'

~/anaconda3/envs/westpa-2020/westpa-2020.03/lib/west_tools/westtools/core.py in main(self)
    156         '''A convenience function to make a parser, parse and process arguments, then run self.go() in the master process.'''
    157         self.make_parser_and_process()
--> 158         with self.work_manager:
    159             if self.work_manager.is_master:
    160                 self.go()

~/anaconda3/envs/westpa-2020/westpa-2020.03/lib/wwmgr/work_managers/core.py in __enter__(self)
     33 
     34     def __enter__(self):
---> 35         self.startup()
     36         return self
     37 

~/anaconda3/envs/westpa-2020/westpa-2020.03/lib/wwmgr/work_managers/processes.py in startup(self)
     97             for iworker,worker in enumerate(self.workers):
     98                 os.environ[pi_name] = str(iworker)
---> 99                 worker.start()
    100             try:
    101                 del os.environ[pi_name]

~/anaconda3/envs/westpa-2020/lib/python3.8/multiprocessing/process.py in start(self)
    119                'daemonic processes are not allowed to have children'
    120         _cleanup()
--> 121         self._popen = self._Popen(self)
    122         self._sentinel = self._popen.sentinel
    123         # Avoid a refcycle if the target function holds an indirect

~/anaconda3/envs/westpa-2020/lib/python3.8/multiprocessing/context.py in _Popen(process_obj)
    222     @staticmethod
    223     def _Popen(process_obj):
--> 224         return _default_context.get_context().Process._Popen(process_obj)
    225 
    226 class DefaultContext(BaseContext):

~/anaconda3/envs/westpa-2020/lib/python3.8/multiprocessing/context.py in _Popen(process_obj)
    281         def _Popen(process_obj):
    282             from .popen_spawn_posix import Popen
--> 283             return Popen(process_obj)
    284 
    285     class ForkServerProcess(process.BaseProcess):

~/anaconda3/envs/westpa-2020/lib/python3.8/multiprocessing/popen_spawn_posix.py in __init__(self, process_obj)
     30     def __init__(self, process_obj):
     31         self._fds = []
---> 32         super().__init__(process_obj)
     33 
     34     def duplicate_for_child(self, fd):

~/anaconda3/envs/westpa-2020/lib/python3.8/multiprocessing/popen_fork.py in __init__(self, process_obj)
     17         self.returncode = None
     18         self.finalizer = None
---> 19         self._launch(process_obj)
     20 
     21     def duplicate_for_child(self, fd):

~/anaconda3/envs/westpa-2020/lib/python3.8/multiprocessing/popen_spawn_posix.py in _launch(self, process_obj)
     45         try:
     46             reduction.dump(prep_data, fp)
---> 47             reduction.dump(process_obj, fp)
     48         finally:
     49             set_spawning_popen(None)

~/anaconda3/envs/westpa-2020/lib/python3.8/multiprocessing/reduction.py in dump(obj, file, protocol)
     58 def dump(obj, file, protocol=None):
     59     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60     ForkingPickler(file, protocol).dump(obj)
     61 
     62 #

TypeError: cannot pickle 'weakref' object

[jdrusso]

Just a thought as I dig into this -- worth mentioning that I'm running MacOS Catalina, which I've bumped into problems with the multiprocessing library before

[jdrusso]

As a hack, since I'm not sure how else to change the default WM w_ipa calls in Jupyter, changing line 27 in the work manager environment.py from 'processes' to 'threads' appears to work. Obviously, this will 'work' at the cost of no longer using processes, and using threads for parallel work instead.

If this hasn't been reported on other platforms, then I'm indeed inclined to say it's related to running the multiprocessing library on Catalina. I'm able to successfully use processes to run the actual simulation.

The environment variable WM_WORK_MANAGER appears to not be used for selecting the w_ipa/WIPI work manager, since that had been set to serial.

[synapticarbors]

I'm wondering if this is a mac + python 3.8 issue. If I run the test suite (new restructured version of the code) using python 3.7 on my Mac (MacOS 10.14.6), all of the process work manager tests fail. But if I use an environment built on python 3.8, I get a bunch of error like you see (TypeError: cannot pickle 'weakref' object). These same test pass under python 3.8 on the linux platform that runs the tests through the Github Actions.

Unfortunately I don't have a solution for this at the moment. It's probably worth it to dig around and see if there are other reports of this in other projects. Maybe they've already shown how to fix it.

[jeremyleung521]

This should no longer be an issue.