misp-modules
misp-modules copied to clipboard
MISP
CSV Import - 'NoneType' object is not iterable
When trying to import a CSV I get the following error:
Apr 19 15:08:34 dsgnmisp01 misp-modules: 2018-04-19 15:08:34,735 - misp-modules - ERROR - Something went wrong:
Apr 19 15:08:34 dsgnmisp01 misp-modules: Traceback (most recent call last):
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 197, in post
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = yield tornado.gen.with_timeout(timeout, self.run_request(jsonpayload))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/site-packages/tornado/gen.py", line 1099, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: value = future.result()
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/asyncio/futures.py", line 274, in result
Apr 19 15:08:34 dsgnmisp01 misp-modules: raise self._exception
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/concurrent/futures/thread.py", line 54, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: result = self.fn(*self.args, **self.kwargs)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 185, in run_request
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = mhandlers[x['module']].handler(q=jsonpayload)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 117, in handler
Apr 19 15:08:34 dsgnmisp01 misp-modules: csv_parser.parse_data(data.split('\n'))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 28, in parse_data
Apr 19 15:08:34 dsgnmisp01 misp-modules: self.delimiter, self.length = self.findDelimiter()
Apr 19 15:08:34 dsgnmisp01 misp-modules: TypeError: 'NoneType' object is not iterable
Apr 19 15:08:34 dsgnmisp01 misp-modules: ERROR:misp-modules:Something went wrong:
Apr 19 15:08:34 dsgnmisp01 misp-modules: Traceback (most recent call last):
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 197, in post
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = yield tornado.gen.with_timeout(timeout, self.run_request(jsonpayload))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/site-packages/tornado/gen.py", line 1099, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: value = future.result()
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/asyncio/futures.py", line 274, in result
Apr 19 15:08:34 dsgnmisp01 misp-modules: raise self._exception
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib64/python3.4/concurrent/futures/thread.py", line 54, in run
Apr 19 15:08:34 dsgnmisp01 misp-modules: result = self.fn(*self.args, **self.kwargs)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "/usr/lib/python3.4/site-packages/misp_modules/__init__.py", line 185, in run_request
Apr 19 15:08:34 dsgnmisp01 misp-modules: response = mhandlers[x['module']].handler(q=jsonpayload)
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 117, in handler
Apr 19 15:08:34 dsgnmisp01 misp-modules: csv_parser.parse_data(data.split('\n'))
Apr 19 15:08:34 dsgnmisp01 misp-modules: File "modules/import_mod/csvimport.py", line 28, in parse_data
Apr 19 15:08:34 dsgnmisp01 misp-modules: self.delimiter, self.length = self.findDelimiter()
Apr 19 15:08:34 dsgnmisp01 misp-modules: TypeError: 'NoneType' object is not iterable
It seems something inside findDelimiter() is causing the issue.
Does someone have a working test.csv (with corresponding headers) I could troubleshoot with?
Well, I never had this kind of issue, do you have the file that causes this error so I can have a look on what is going on ?
I'm trying it with iSight-provided CSV's. Below is an example that is failing on me...
I'm sure there's something messed up with the headers because they're not all valid MISP attribute names, but the error it's throwing makes me think it doesn't even get to the attribute-checking part before it fails.
reportId,title,ThreatScape,audience,intelligenceType,publishDate,reportLink,webLink,emailIdentifier,senderAddress,senderName,sourceDomain,sourceIp,subject,recipient,emailLanguage,fileName,fileSize,fuzzyHash,fileIdentifier,md5,sha1,sha256,description,fileType,packer,userAgent,registry,fileCompilationDateTime,filePath,asn,cidr,domain,domainTimeOfLookup,networkIdentifier,ip,port,protocol,registrantEmail,registrantName,networkType,url,malwareFamily,malwareFamilyId,actor,actorId,observationTime
14-00000132,"Neverquest_ Behavior, Capabilities and Communications","Cyber Crime,Enterprise",Operational,threat,1418772480,https://api.isightpartners.com/report/14-00000132,https://mysight.isightpartners.com/report/full/14-00000132,,,,,,,,,,,,,,,,,,,,,,,,,,,Attacker,,,,,,ControlServer,http://drevprom.ru/data/feeder,vawtrak,4b701250-266d-4eb3-9a04-adef2e83c031,,,1469947711
14-00000132,"Neverquest_ Behavior, Capabilities and Communications","Cyber Crime,Enterprise",Operational,threat,1418772480,https://api.isightpartners.com/report/14-00000132,https://mysight.isightpartners.com/report/full/14-00000132,,,,,,,,,,,,,,,,,,,,,,,,,,,Attacker,,,,,,ControlServer,http://rusemp2894.ru/data/feeder,vawtrak,4b701250-266d-4eb3-9a04-adef2e83c031,,,1469947710
I updated the module to have the header in user configuration side. You can find instructions about the way to use it in Pull request #190
Don't hesitate to ping me for any further question