JamSpell icon indicating copy to clipboard operation
JamSpell copied to clipboard

Published 20 hours ago • verified publisher icon bakwc

Memory leak on corrector.LoadLangModel()

Open rominf opened this issue 5 years ago • 3 comments

I've trained my model (I've tried versions from master and 0.0.11 branches) on 10 MiB plain text part of English Wikipedia (enwiki-latest-pages-articles_10MiB.txt) and got 41 MiB bin file (enwiki.bin.zip).

I'm loading it in Python, but it takes 12 GiB of memory to load it and still it doesn't load in foreseeable time.

rominf avatar Aug 22 '18 08:08 rominf

I'm facing the same issue. I've trained a (german) model using around 560MiB plain text from the Leipzig Corpora Collection. The model itself is 488MiB. Having a 16GB RAM 4CPU Linux Cloud, it takes 5-10 min to load the model. Is it possible to speed this up?

xdingo93x avatar Sep 28 '18 11:09 xdingo93x

March 16, 2022 I tried to use this on Windows 10, Python 3.9.7 but it keeps eating memory and nothing happens. Unable to load the 34 MB English model.

mshakirDr avatar Mar 16 '22 09:03 mshakirDr

Community version doesn't support loading linux models on windows. You should either use a linux or train your own model on windows.

Or you can buy a PRO version, it supports all models on all operation systems. Also a PRO version reduced memory usage while training.

bakwc avatar Mar 16 '22 09:03 bakwc