taranis-ai
taranis-ai copied to clipboard
taranis-ai
Update MISP Presenter template
From Taranis-NG created by ewoiset: ait-cs-IaaS/Taranis-NG#91
Review and update misp template in presenter, to export all relevant information - tags, topics, etc...
Using https://pymisp.readthedocs.io/en/latest/ in https://github.com/ait-cs-IaaS/Taranis-NG/blob/awake/src/worker/worker/publishers/misp_publisher.py to publish a json created by:
https://github.com/ait-cs-IaaS/Taranis-NG/blob/awake/src/core/core/managers/workers_pre_seed.py#L286C6-L286C6
with this template: https://github.com/ait-cs-IaaS/Taranis-NG/blob/awake/src/core/core/static/presenter_templates/misp_template.json
So the main change for this issue is to adapt the misp_template.json to include the "relevant information".
See https://www.misp-project.org/openapi/#tag/Events/operation/addEvent for available fields
Adding the actual stories would be a good first improvement
{% for story in report_item.news_item_aggregates %}
{{ story }}
{% endfor %}
The initial thought of mapping our data to plain attributes in events seems not to be the most effective approach. => Therefore I made a mock up of how the mapping to MISP could look like.
I tried to map data from news-item-aggregate_* testdata.py
To not make things complex, a well-fitting MISPObject was picked (Vulnerability).
The initial event data:
type = "MISP_PUBLISHER"
name = "MISP Publisher"
description = "Publisher for publishing in MISP"
# MISP BASE_URL fix https://github.com/MISP/misp-docker/issues/130#issuecomment-1033312591
misp_url = "https://localhost"
misp_verifycert = False
misp_key = <key>
event_json = {
"org_id": "1231231231416",
"distribution": "0",
"info": "wow_with_attribute",
"orgc_id": "123452",
# "uuid": "c99506a6-1255-4b71-afa5-7b8ba48c3b11",
"date": "1991-01-15",
"published": False,
"analysis": "0",
"attribute_count": "321",
"timestamp": "1617875569",
"sharing_group_id": "1",
"proposal_email_lock": True,
"locked": True,
"threat_level_id": "1",
"publish_timestamp": "1617875569",
"sighting_timestamp": "1617875569",
"disable_correlation": False,
# "extends_uuid": "c99506a6-1255-4b71-afa5-7b8ba48c3b11",
"event_creator_email": "[email protected]",
"Attribute": [{
"category": "External analysis",
"type": "text",
"value": "This is a story",
"to_ids": False,
"uuid": "c99506a6-1255-4b71-afa5-7b8ba48c3b1s",
"timestamp": "1617875569",
"distribution": "0",
"sharing_group_id": "1",
"comment": "logged source ip",
"deleted": False,
"disable_correlation": True
}]
}
Creation of the event.
misp = ExpandedPyMISP(misp_url, misp_key, misp_verifycert)
event = MISPEvent()
event.load(event_json)
Keys in JSON need to be adjusted to match the Vulnerability MISPObject.
# @description is "title"
# @references is "link"
# @summary is "content"
# @published stays unchanged
object_json = {
"description": "Test News Item 14",
"references": "https://url/13",
"summary": "CVE-2020-1234 - Test Aggregate 1",
"published": "2023-08-01T17:01:04.801998"
}
Importing to MISP instance could look like this:
misp_object = event.add_object(name='vulnerability')
for key, value in object_json.items():
misp_object.add_attribute(key, value=value)
misp.add_event(event)
or like this:
misp_object = event.add_object(name='vulnerability')
obj_attr = misp_object.add_attribute('description', value=object_json['description'])
misp_object.add_attribute('summary', value=object_json['summary'])
misp_object.add_attribute('references', value=object_json['summary'])
misp_object.add_attribute('published', value=object_json['published'])
misp.add_event(event)
Ad.1 - I was not able to load the JSON data without the FOR loop. Ad.2 - From what I understood, it should be able to create custom MISPObjects dynamically without a template (Did not make it work.). Nevertheless, without an object template in the MISP instance, the object might not be possible to edit in the UI. Ad.3 - Using fields of the Vulnerability object, stories can be added to the text fields.
That looks like a good start! To increase reuseability and readability I would suggest creating our own MISPObject the
class TaranisObejct(AbstractMISPObjectGenerator):
Looking something like:
https://github.com/MISP/PyMISP/blob/main/pymisp/tools/urlobject.py or https://github.com/MISP/PyMISP/blob/main/pymisp/tools/fail2banobject.py
These Objects could than be created and added in a loop to the event:
https://github.com/MISP/PyMISP/blob/3a3096664109639322bff7285956ef7cc376a260/examples/load_csv.py#L74
The event JSON could be simplified by removing the now obsolete attributes since this would represented in the TaranisObject.
The base class for TaranisObject:
class TaranisObject(MISPObject):
def __init__(self, parameters: dict, strict: bool = True, **kwargs):
super().__init__(name='taranis-news-item', strict=strict, **kwargs)
self._parameters = parameters
self.generate_attributes()
def generate_attributes(self):
"""Contains the logic where all the values of the object are gathered"""
if hasattr(self, '_parameters'):
for object_relation in self._definition['attributes']:
value = self._parameters.pop(object_relation, None)
if not value:
continue
if isinstance(value, dict):
self.add_attribute(object_relation, **value)
elif isinstance(value, list):
self.add_attributes(object_relation, *value)
else:
# Assume it is the value only
self.add_attribute(object_relation, value=value)
if self._strict and self._known_template and self._parameters:
raise InvalidMISPObject(
'Some object relations are unknown in the template and could not be attached: {}'.format(
', '.join(self._parameters)))
Test scenario:
type = "MISP_PUBLISHER"
name = "MISP Publisher"
description = "Publisher for publishing in MISP"
misp_url = "https://localhost"
misp_verifycert = False
misp_key = <key>
event_json = {
# "distribution": "0", # this attribute might be interesting
"info": "Event with Taranis objects",
# "orgc_id": "taranis_id", # this attribute could be interesting
"disable_correlation": False
}
stories = [
{
"title": "Test News Item 1",
"link": "https://url/13",
"content": "CVE-2020-1234 - Test Aggregate 1",
"published": "2023-08-01T17:01:04.801998"
},
{
"title": "Test News Item 2",
"link": "https://url/13",
"content": "CVE-2020-1234 - Test Aggregate 1",
"published": "2023-08-01T17:01:04.801998"
},
{
"title": "Test News Item 3",
"link": "https://url/13",
"content": "CVE-2020-1234 - Test Aggregate 1",
"published": "2023-08-01T17:01:04.801998"
},
{
"title": "Test News Item 4",
"link": "https://url/13",
"content": "CVE-2020-1234 - Test Aggregate 1",
"published": "2023-08-01T17:01:04.801998"
},
{
"title": "Test News Item 5",
"link": "https://url/13",
"content": "CVE-2020-1234 - Test Aggregate 1",
"published": "2023-08-01T17:01:04.801998"
}
]
misp = ExpandedPyMISP(misp_url, misp_key, misp_verifycert)
event = MISPEvent()
event.load(event_json)
for story in stories:
taranis_obj = TaranisObject(parameters=story)
event.add_object(taranis_obj)
misp.add_event(event)
taranis-news-item template:
{
"attributes": {
"title": {
"description": "Title of the Taranis news item",
"misp-attribute": "text",
"ui-priority": 0
},
"link": {
"description": "URL address to the Taranis news item source.",
"misp-attribute": "link",
"ui-priority": 0
},
"content": {
"description": "Content of the Taranis news item",
"misp-attribute": "text",
"multiple": true,
"ui-priority": 0
},
"published": {
"description": "Publish date of the Taranis news item ",
"misp-attribute": "datetime",
"ui-priority": 0
}
},
"description": "An object describing a news item from Taranis AI",
"meta-category": "misc",
"name": "taranis",
"uuid": "e4b7e5e3-2c4c-4edf-b9a3-2c2a32844e61",
"version": 1
}
This way the MISP template is already in use (solely for PyMISP, not for the MISP instance per see).
See the whole mock-up: https://github.com/not4Pedro/misp-implementation
@b3n4kh what do you think?