Improve MsgParser analyzer adding attachment and IOC as observables

[Waltyon]

Hello, I was inspired by EmlParser to improve MsgParser. It integrates attachments as observables as well as possible IOC.

• Add refactor modules by using external libraries
• Explore attachments to import into observables and the lists with their hashes (md5, sha1, sha256)
• Search for possible IOC in the mail (IP, Hash, url, email address..)
• The code structure was inspired by EmlParser analyzer

[azgaviperr]

Gonna try that tomorrow

[dadokkio]

I made some tests with random mails I had on my machine. 2 of them failed for reasons related to embedded attachments:

Traceback (most recent call last): 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 116, in <module> MsgParserAnalyzer().run() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 110, in run parsingResult = self.parseMsg() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 92, in parseMsg attachment_name = '{}/{}'.format(str(self.temp_dir.name), str(an_attachment.filename)) 
File "/usr/local/lib/python3.7/dist-packages/outlook_msg/attachment.py", line 23, in filename return self.mfs['PidTagAttachLongFilename'] 
File "/usr/local/lib/python3.7/dist-packages/outlook_msg/message_file_storage.py", line 104, in __getitem__ raise KeyError(item)
KeyError: 'PidTagAttachLongFilename'

Traceback (most recent call last): 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 116, in <module> MsgParserAnalyzer().run() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 110, in run parsingResult = self.parseMsg() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 95, in parseMsg with an_attachment.open() as attachment_fp, open(attachment_name, 'wb') as output_fp: 
File "/usr/local/lib/python3.7/dist-packages/outlook_msg/attachment.py", line 15, in open raise NotImplementedError(f"Unable to open attachments stored as: {self.attachment_method.name}")
NotImplementedError: Unable to open attachments stored as: EmbeddedMessage

For the remaining ones everything was fine. All attachments, emails and hashes have been added properly to the observable :+1:

[Waltyon]

I made some tests with random mails I had on my machine. 2 of them failed for reasons related to embedded attachments:

Traceback (most recent call last): 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 116, in <module> MsgParserAnalyzer().run() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 110, in run parsingResult = self.parseMsg() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 92, in parseMsg attachment_name = '{}/{}'.format(str(self.temp_dir.name), str(an_attachment.filename)) 
File "/usr/local/lib/python3.7/dist-packages/outlook_msg/attachment.py", line 23, in filename return self.mfs['PidTagAttachLongFilename'] 
File "/usr/local/lib/python3.7/dist-packages/outlook_msg/message_file_storage.py", line 104, in __getitem__ raise KeyError(item)
KeyError: 'PidTagAttachLongFilename'

Traceback (most recent call last): 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 116, in <module> MsgParserAnalyzer().run() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 110, in run parsingResult = self.parseMsg() 
File "/opt/cortex/analyzers/MsgParser/parse.py", line 95, in parseMsg with an_attachment.open() as attachment_fp, open(attachment_name, 'wb') as output_fp: 
File "/usr/local/lib/python3.7/dist-packages/outlook_msg/attachment.py", line 15, in open raise NotImplementedError(f"Unable to open attachments stored as: {self.attachment_method.name}")
NotImplementedError: Unable to open attachments stored as: EmbeddedMessage

For the remaining ones everything was fine. All attachments, emails and hashes have been added properly to the observable 👍

Hello @dadokkio,

Would it be possible to have the .msg's that have an issue so I can see if I can fix it? Unfortunately I have not been able to reproduce the problem with embedded attachments

Thanks