binaryalert
binaryalert copied to clipboard
Create a different queue for retroactive analysis
Background
Related to #102 , it's always felt awkward to have the same SQS queue for both regular operation and for retroactive analysis:
- New binaries added to the BinaryAlert bucket might not be scanned for hours if a large retro scan is underway
- A
live_test
will fail for the same reason if a retro scan is running - A
purge_queue
to stop a runaway retro scan will also drop new object events from the queue - The number of concurrent analyzers needed for a retro scan vs. normal operation could be vastly different
These problems could all be fixed if there were just a different SQS queue for retroactive analysis!
Desired Change
Create a new SQS queue specifically for retro scans. It's not clear whether Lambda will allow 2 different SQS event source mappings, but it's worth a shot.
On top of this, might it be helpful to publish retrospective no_yara_matches to a different SNS topic?
Background behind that is, users might have a flow whereby they subscribe to no_yara_match for continued file processing. If retrospective scanning were to publish messages on that topic once again, they could be double-processed.