presidio icon indicating copy to clipboard operation
presidio copied to clipboard

add support for multiprocessing to spacy pipelines

Open skunkwerk opened this issue 6 months ago • 1 comments

Is your feature request related to a problem? Please describe. The BatchAnalyzerEngine which calls the Spacy engine's process_batch method doesn't seem to pass through any kwargs for setting the n_process or batch size.

Describe the solution you'd like We should update process_batch to pass through kwargs to spacy's pipe method.

Describe alternatives you've considered Haven't considered any.

Additional context N/A

skunkwerk avatar Aug 20 '24 22:08 skunkwerk