python-crfsuite icon indicating copy to clipboard operation
python-crfsuite copied to clipboard

Published 20 hours ago • verified publisher icon scrapinghub

Training Issues in a Multi-Threaded Environment

Open peblair opened this issue 5 years ago • 1 comments

We are using this library in a multi-threaded application, and we have run into an issue with thread-friendliness. When the library drops down into C++ and trains the model, it does not appear to be releasing the GIL. This locks up the interpreter (meaning other threads are halted) until training finishes. Would it be possible to tweak the library such that, while executing the crfsuite code, the GIL is released? Thanks!

peblair avatar Mar 20 '20 08:03 peblair

It should be possible to do so, but I haven't checked it in detail. In the meantime, you can also use processes.

kmike avatar Mar 25 '20 20:03 kmike