MISP-tools
MISP-tools copied to clipboard
CrowdStrike
Adversary Import fails with Code 500
I can import reports and but importing adversaries fails everytime:
[2025-01-14 16:39:17,544] INFO processor/main _____ _______ _____ _____ ______ _______
[2025-01-14 16:39:17,544] INFO processor/main | | | | |_____] | | |_____/ |
[2025-01-14 16:39:17,544] INFO processor/main __|__ | | | | |_____| | \_ |
[2025-01-14 16:39:17,544] INFO processor/main
[2025-01-14 16:39:17,544] INFO processor/main
[2025-01-14 16:39:17,544] INFO processor/main ____ ___ __ __ ___ ____ _____ ____ ____ ____ ___ _____
[2025-01-14 16:39:17,544] INFO processor/main / T| \ | T | / _]| \ / ___/ / T| \ l j / _]/ ___/
[2025-01-14 16:39:17,544] INFO processor/main Y o || \ | | | / [_ | D )( \_ Y o || D ) | T / [_( \_
[2025-01-14 16:39:17,544] INFO processor/main | || D Y| | |Y _]| / \__ T| || / | | Y _]\__ T
[2025-01-14 16:39:17,544] INFO processor/main | _ || |l : !| [_ | \ / \ || _ || \ | | | [_ / \ |
[2025-01-14 16:39:17,544] INFO processor/main | | || | \ / | T| . Y \ || | || . Y j l | T\ |
[2025-01-14 16:39:17,544] INFO processor/main l__j__jl_____j \_/ l_____jl__j\_j \___jl__j__jl__j\_j|____jl_____j \___j
[2025-01-14 16:39:17,544] INFO processor/main
[2025-01-14 16:39:17,544] INFO processor/main Start Threat Actor galaxy cluster alignment
[2025-01-14 16:39:17,545] INFO processor/main Retrieving all adversaries.
[2025-01-14 16:39:18,057] INFO processor/main Got 257 adversaries from the Crowdstrike Intel API.
[2025-01-14 16:39:18,451] INFO processor/main Retrieving all adversaries.
Traceback (most recent call last):
File "MISP-tools/misp_import.py", line 505, in <module>
main()
File "/MISP-tools/misp_import.py", line 497, in main
import_handler.build()
File "/MISP-tools/misp_import.py", line 401, in build
self.import_new_events()
File "/MISP-tools/misp_import.py", line 387, in import_new_events
self.importer.import_from_crowdstrike(
File "/MISP-tools/cs_misp_import/importer.py", line 342, in import_from_crowdstrike
self.actors_importer.process_actors(actors_days_before, self.event_ids)
File "/MISP-tools/cs_misp_import/actors.py", line 184, in process_actors
cluster_result = self.misp.add_galaxy_cluster(get_threat_actor_galaxy_id(self.misp), cluster)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/misp-modules/lib/python3.11/site-packages/pymisp/api.py", line 1825, in add_galaxy_cluster
cluster_j = self._check_json_response(r)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/misp-modules/lib/python3.11/site-packages/pymisp/api.py", line 3978, in _check_json_response
r = self._check_response(response, expect_json=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/MISP-tools/cs_misp_import/misp_safe_check_response.py", line 55, in safe_check_response
raise MISPServerError(fail_msg)
pymisp.exceptions.MISPServerError: Error code 500: SQLSTATE[01000]: Warning: 1265 Data truncated for column 'galaxy_id' at row 1
In the MISP error.log I see:
2025-01-14 15:34:03 Error: [PDOException] SQLSTATE[01000]: Warning: 1265 Data truncated for column 'galaxy_id' at row 1
Request URL: /galaxy_clusters/add/698774c7-8022-42c4-917f-8d6e4f06ada3
Stack Trace:
#0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(502): PDOStatement->execute()
#1 /var/www/MISP/app/Model/Datasource/Database/MysqlObserverExtended.php(162): DboSource->_execute()
#2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(1132): MysqlObserverExtended->execute()
#3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1943): DboSource->create()
#4 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1761): Model->_doSave()
#5 /var/www/MISP/app/Model/GalaxyCluster.php(342): Model->save()
#6 /var/www/MISP/app/Controller/GalaxyClustersController.php(321): GalaxyCluster->saveCluster()
#7 [internal function]: GalaxyClustersController->add()
#8 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(500): ReflectionMethod->invokeArgs()
#9 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction()
#10 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()
#11 /var/www/MISP/app/webroot/index.php(107): Dispatcher->dispatch()
#12 {main}
I also got similar kind of error but when importing indicators:
[2025-01-15 15:21:26,298] INFO config _______ _ _ _______ _______ _ _ _______ _____ __ _ _______ _____ ______
[2025-01-15 15:21:26,298] INFO config | |_____| |______ | |____/ | | | | \ | |______ | | ____
[2025-01-15 15:21:26,298] INFO config |_____ | | |______ |_____ | \_ |_____ |_____| | \_| | __|__ |_____|
[2025-01-15 15:21:26,298] INFO config
[2025-01-15 15:21:26,299] WARNING config misp_enable_ssl SSL is disabled for MISP API requests
[2025-01-15 15:21:27,024] INFO config No configuration errors found (1 warning)
[2025-01-15 15:21:27,024] INFO config
[2025-01-15 15:21:27,024] INFO config ____ _ _ ____ ____ _ _ ____ ___ ____ ____ ____ ____ ___
[2025-01-15 15:21:27,024] INFO config | |__| |___ | |_/ [__ |__] |__| [__ [__ |___ | \
[2025-01-15 15:21:27,024] INFO config |___ | | |___ |___ | \_ ___] | | | ___] ___] |___ |__/
[2025-01-15 15:21:27,024] INFO config
/home/misp/csenv/lib/python3.10/site-packages/pymisp/__init__.py:67: FutureWarning: This class is deprecated, use PyMISP instead
warnings.warn('This class is deprecated, use PyMISP instead', FutureWarning)
[2025-01-15 15:21:28,066] INFO processor/main Retrieving all galaxy cluster values for the Android cluster.
Traceback (most recent call last):
File "/home/misp/MISP-tools/misp_import.py", line 505, in <module>
main()
File "/home/misp/MISP-tools/misp_import.py", line 495, in main
import_handler = ImportHandler(config, intel_api_client,
File "/home/misp/MISP-tools/misp_import.py", line 312, in __init__
self.importer = CrowdstrikeToMISPImporter(
File "/home/misp/MISP-tools/cs_misp_import/importer.py", line 76, in __init__
self.all_galaxies = self.get_galaxies()
File "/home/misp/MISP-tools/cs_misp_import/importer.py", line 270, in get_galaxies
all_galaxies.append(self.misp_client.search_galaxy_clusters(gal["id"], searchall=""))
File "/home/misp/csenv/lib/python3.10/site-packages/pymisp/api.py", line 1781, in search_galaxy_clusters
clusters_j = self._check_json_response(r)
File "/home/misp/csenv/lib/python3.10/site-packages/pymisp/api.py", line 3978, in _check_json_response
r = self._check_response(response, expect_json=True)
File "/home/misp/MISP-tools/cs_misp_import/misp_safe_check_response.py", line 55, in safe_check_response
raise MISPServerError(fail_msg)
pymisp.exceptions.MISPServerError: Error code 500: An Internal Error Has Occurred.
My MISP error.log shows:
2025-01-15 15:21:28 Error: [PDOException] SQLSTATE[42S22]: Column not found: 1054 Unknown column 'Galaxy.default' in 'where clause'
Request URL: /galaxy_clusters/index/3
Stack Trace:
#0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(502): PDOStatement->execute()
#1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(468): DboSource->_execute()
#2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(715): DboSource->execute()
#3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(1226): DboSource->fetchAll()
#4 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(3053): DboSource->read()
#5 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(3025): Model->_readDataSource()
#6 /var/www/MISP/app/Model/AppModel.php(4359): Model->find()
#7 /var/www/MISP/app/Controller/GalaxyClustersController.php(105): AppModel->find()
#8 [internal function]: GalaxyClustersController->index()
#9 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(499): ReflectionMethod->invokeArgs()
#10 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction()
#11 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()
#12 /var/www/MISP/app/webroot/index.php(105): Dispatcher->dispatch()
#13 {main}
I realize this happen after I upgraded my MISP version to 2.4.200
I also got similar kind of error but when importing indicators:
I realize this happen after I upgraded my MISP version to 2.4.200
Actually, your specific issue is also reported in the MISP repository: https://github.com/MISP/MISP/issues/10062 I don't know how hard it is to patch the SQL query yourself, but good luck :-)
Fixed my issue by editing line 169 in cs_misp_import/helper.py and setting ta_galaxy_id = gal["Galaxy"]["uuid"] to ta_galaxy_id = gal["Galaxy"]["id"]
The error suggested that the Galaxy ID used to create the GalaxyCluster was too long.
Taking a look at the galaxy_clusters table, a Galaxy ID was max. 11 integers, but when debugging I saw a UUID being used to add the GalaxyCluster.
Created Pull request 192.
As described in the following MISP PR, in v2.4.197 to v2.4.204 of MISP, API calls to add a GalaxyCluster to a Galaxy using a Galaxy's uuid would fail (https://github.com/MISP/MISP/pull/10221).
The cs_misp_import/actors.py creates GalaxyClusters to represent Actors/Adversaries and adds those clusters to the Threat Actors Galaxy in MISP using a uuid as a reference to the Galaxy.
Fixed/bypassed by PR: #192
