acm-sdk-python
acm-sdk-python copied to clipboard
解决windows下add_watcher报can't pickle _thread.RLock错误
Windows下执行add_watcher方法会报如下错误(linux正常): """ Traceback (most recent call last): File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\site-packages\acm\commons.py", line 10, in synced_func return func(*args, **kws) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\site-packages\acm\client.py", line 557, in add_watcher self.add_watchers(data_id, group, [cb]) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\site-packages\acm\commons.py", line 10, in synced_func return func(*args, **kws) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\site-packages\acm\client.py", line 606, in add_watchers puller.start() File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\multiprocessing\popen_spawn_win32.py", line 65, in init reduction.dump(process_obj, to_child) File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: can't pickle _thread.RLock objects python-BaseException Traceback (most recent call last): File "C:\Users\lenovo\AppData\Local\Programs\Python\Python36\lib\multiprocessing\spawn.py", line 115, in _main self = reduction.pickle.load(from_parent) EOFError: Ran out of input python-BaseException """ 可能是因为_do_pulling方法内部使用了ACMClient类型的self对象,这个类里面有几个成员是threading.RLock类型的和Process一起用会有些问题。 相关参考: https://stackoverflow.com/questions/7865430/multiprocessing-pool-picklingerror-cant-pickle-type-thread-lock-attribu https://stackoverflow.com/questions/44144584/typeerror-cant-pickle-thread-lock-objects 两种方法可解决这个问题:
- 将_do_pulling不作为类成员函数,作为独立的函数,相关需要的数据作为参数传入。改动大些。
- 将_do_pulling从Process中执行改为Thread中执行。 这里按第二种方法改的,也是考虑到_do_pulling中多是IO操作,不是CPU密集型任务,不需要创建进程,线程更合适些。
里面的process_mgr是多个线程的共享列表,直接用原生的list就可以实现,是线程安全的,所以这里去掉了Manager().list
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.