Cortex-Analyzers icon indicating copy to clipboard operation
Cortex-Analyzers copied to clipboard

opencti performance improvement for exact searches

Open gelim opened this issue 3 years ago • 0 comments

Hello,

Please find a proposal for improving the observables exact search by using the .read() API call with a proper filters set. A dictionary is used with main Cortex observable types to be translated into OpenCTI key value.

For the moment it supports the following Cortex types: ip, url, domain, mail, hash and filename. As well the helper function get_hash_type() has been added to do a regex match on Cortex type hash and determine if it's an MD5, a SHA1 or a SH256 in order to translate to the proper OpenCTI observable type.

For any observable type that is not present in the cortex2opencti_types full text search (slow) will be used. Performance here without the patch for querying one observable is ~10sec, with the patch it is under the second.

Without this, doing batch analyzer runs is prone to failure as the connection between Cortex instance and OpenCTI will easily timeout.

Cheers, -- Mathieu

gelim avatar Oct 12 '21 15:10 gelim