ltp icon indicating copy to clipboard operation
ltp copied to clipboard

Abnormal behavior with the multiprocessing package

Open linfeng-du opened this issue 4 years ago • 1 comments

I have tried on multiple machines and it seems that when adding the importation code, running Pool workers will take up a huge amount of memory and very likely to cause an OOM :( Even for the simple code snippet as below (the importation code itself doesn't seem to be very memory consuming).

from multiprocessing import Pool
from ltp import LTP


def f(x):
    return x * x


if __name__ == '__main__':
    with Pool() as pool:
        pool.map(f, range(10))

I'm wondering if you could reproduce this problem 0.0.

linfeng-du avatar Jan 25 '21 17:01 linfeng-du

I test your code and reproduce the problem and I will fix it soon. Hope everything goes well. WeChat450c7a6e1f515cd2f9dbc4a3b637a02e

AlongWY avatar Feb 27 '21 04:02 AlongWY