streamalert icon indicating copy to clipboard operation
streamalert copied to clipboard

Published 20 hours ago • verified publisher icon airbnb

Improvement: Multiprocessing of classification

Open jacknagz opened this issue 7 years ago • 1 comments

Background

AWS Lambda supports a thread limit of 1024 per execution. We currently are not taking advantage of this feature when processing logs with StreamAlert.

Desired Outcome

Usage of Python's multiprocessing builtin library to parallelize the classification and rules matching engine.

jacknagz avatar Jul 26 '17 18:07 jacknagz

PR #273 is open with a POC for this but should be further investigated.

ryandeivert avatar Sep 08 '17 16:09 ryandeivert