intelmq
intelmq copied to clipboard
certtools
IntelMQ is a solution for IT security teams for collecting and processing security feeds using a message queuing protocol.
When CSV parser parses the file and event line contains time format like %d-%m-%Y, fuzzy time conversion fails to guess correct time format. Example of the feed is http://benkow.cc/export.php Fuzzy...
This parser can translate json key into harmonized key field and can create events from list of dict within key of JSON.
Several expert bots have the capability, to update their needed local copy of an external database. They update-routine currently checks, if any such bot is configured before starting actually: https://github.com/certtools/intelmq/blob/9eb388e8f56b5844c86f9da9d1fe8acd8fecfa63/intelmq/bots/experts/tor_nodes/expert.py#L79-L90...
E.g.: - CSV dialect - Quoting - Escaping just the standard options of Python's CSV implementation
https://github.com/certtools/intelmq/actions/runs/1191112061 > Annotations > Analyze (python) > 2 issues were detected with this workflow: git checkout HEAD^2 is no longer necessary. Please remove this step as Code Scanning recommends analyzing...
These regular expressions should be compiled using `re.compile`: https://github.com/certtools/intelmq/blob/420a8d454db3cf4c7bfa0c4cd17a756b0cb3dbc3/intelmq/bots/parsers/danger_rulez/parser.py#L7-L8
An expert to fetch the HTTP (or more protocols?) header/content from a URL, e.g. `source.url`. Could be processed for filtering later.
E.g. this fails: ``` event['feed.accuracy'] *= ioc['confidence']/100 ``` with: ``` intelmq.lib.exceptions.KeyExists: key 'feed.accuracy' already exists ``` as internally the setitem method is called.
Reasoning: if there is data in the postgresql DB which can not be further processed, I'd like to have a feature where I can SELECT (sql select) these rows and...
While observing intelmq-manager and monitoring the throughput of different bots, I noticed that you can only get a snapshot in time of the queue length. However, the snapshot really depends...