lac icon indicating copy to clipboard operation
lac copied to clipboard

python 多进程调用失败

Open CarolsYang opened this issue 3 years ago • 1 comments

# SentenceAnalysis() ## 这里做实例化lac的模型
pool=Pool(processes=30)
res= pool.map(SentenceAnalysis().sentence_struct,querylist) ## sentence_struct就是用的lac来iqu词性、词的重要性等操作

Traceback (most recent call last): File "/mnt/data/yzy/project/DataPrepare/bin/VoiceDemand.py", line 199, in res= pool.map(sentenceanaly.sentence_struct,querylist) File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/pool.py", line 268, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/pool.py", line 657, in get raise self._value File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/pool.py", line 431, in _handle_tasks put(task) File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/home/yiziyang/.conda/envs/nlu_train/lib/python3.7/multiprocessing/reduction.py", line 51, in dumps cls(buf, protocol).dump(obj) TypeError: can't pickle paddle.fluid.core_avx.CPUPlace objects

CarolsYang avatar Nov 24 '22 09:11 CarolsYang

LAC 版本是2.1.2,有没有相应的多进程的例子啊

CarolsYang avatar Nov 24 '22 09:11 CarolsYang