ReDet icon indicating copy to clipboard operation
ReDet copied to clipboard

多线程预测问题,Can't pickle local object 'Cyclic Group.irrep.<locals>.<lambda>'

Open 1994su opened this issue 2 years ago • 3 comments

作者您好,在预测大的遥感影像的时候想使用多线程进行预测,但是e2cnn中的Cyclic Group导致不能调用pool.apply,请问对这方面您有经验吗

1994su avatar Nov 08 '21 06:11 1994su

请附上具体的log

csuhan avatar Nov 27 '21 03:11 csuhan

请附上具体的log

Traceback (most recent call last): File "/home/cresda/ProgramCode/ReDet-master_cuda/demo_large_image_multiProcess.py", line 379, in pool.apply(inference_single, args=(model,classnames,cv_img, stride, chip_size_list, col, row, real_window_col, real_window_row,num)) File "/home/cresda/anaconda3/lib/python3.7/multiprocessing/pool.py", line 261, in apply return self.apply_async(func, args, kwds).get() File "/home/cresda/anaconda3/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value File "/home/cresda/anaconda3/lib/python3.7/multiprocessing/pool.py", line 431, in _handle_tasks put(task) File "/home/cresda/anaconda3/lib/python3.7/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/home/cresda/anaconda3/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps cls(buf, protocol).dump(obj) AttributeError: Can't pickle local object 'CyclicGroup.irrep..' [W CudaIPCTypes.cpp:22] Producer process has been terminated before all shared CUDA tensors released. See Note [Sharing CUDA tensors]

1994su avatar Nov 29 '21 07:11 1994su

参考:https://stackoverflow.com/questions/8804830/python-multiprocessing-picklingerror-cant-pickle-type-function 可能对你的问题有帮助:

I'd use pathos.multiprocesssing, instead of multiprocessing. pathos.multiprocessing is a fork of multiprocessing that uses dill. dill can serialize almost anything in python, so you are able to send a lot more around in parallel. The pathos fork also has the ability to work directly with multiple argument functions, as you need for class methods.

csuhan avatar Nov 29 '21 07:11 csuhan