ltp
ltp copied to clipboard
Abnormal behavior with the multiprocessing package
I have tried on multiple machines and it seems that when adding the importation code, running Pool workers will take up a huge amount of memory and very likely to cause an OOM :( Even for the simple code snippet as below (the importation code itself doesn't seem to be very memory consuming).
from multiprocessing import Pool
from ltp import LTP
def f(x):
return x * x
if __name__ == '__main__':
with Pool() as pool:
pool.map(f, range(10))
I'm wondering if you could reproduce this problem 0.0.
I test your code and reproduce the problem and I will fix it soon.
Hope everything goes well.