connectors
connectors copied to clipboard
OpenCTI-Platform
[Malpedia] Connector creating external reference
Description
I have a docker setup that is running the malpedia connector without any API key. When running the connector to start the import, it will create some malware entities as well as the organisation etc but errors appear in the logs then the connector stops until the next run time or manually restart the docker container.
Environment
- OS (where OpenCTI server runs): Ubuntu 20.10/Docker
- OpenCTI version: 4.5.3
- OpenCTI client: frontend
- Other environment details: none
Reproducible Steps
Steps to create the smallest reproducible scenario:
- Create malpedia connector in docker
- Create user
- Add user key to config
- Launch
Expected Output
Items to be pulled from the malpedia API and enter the information into opencti
Actual Output
` INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["Flame"]}].
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 140
INFO:root:Tag 'Flame' does not reference malware
INFO:root:Listing Malwares with filters [{"key": "name", "values": ["sKyWIper"]}].
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 140
INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["sKyWIper"]}].
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 140
INFO:root:Tag 'sKyWIper' does not reference malware
INFO:root:Listing Malwares with filters [{"key": "name", "values": ["win.flame"]}].
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 140
INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["win.flame"]}].
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 140
INFO:root:Tag 'win.flame' does not reference malware
INFO:root:Reading Marking-Definition {marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9}.
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 502
INFO:root:Creating Malware {Flame}.
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 236
ERROR:root:Restricted entity already exists
ERROR:root:error creating malware entity: {'name': 'UnsupportedError', 'message': 'Restricted entity already exists'}
ERROR:root:some error occurred during malware creation
INFO:root:Processing malware family: win.nagini
INFO:root:Processing malware family: win.nagini
INFO:root:Listing Malwares with filters [{"key": "name", "values": ["Nagini"]}].
INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["Nagini"]}].
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 140
INFO:root:Tag 'Nagini' does not reference malware
INFO:root:Listing Malwares with filters [{"key": "name", "values": ["win.nagini"]}].
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 140
INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["win.nagini"]}].
INFO:root:Tag 'win.nagini' does not reference malware
INFO:root:Reading Marking-Definition {marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9}.
INFO:root:Creating Malware {Nagini}. INFO:root:Creating External Reference {Malpedia}.
INFO:root:Adding External-Reference {05f3bcfa-7a15-4884-8609-92f9f952ee07} to Stix-Domain-Object {2198398d-ceba-46f0-bb9e-9a012158463a}
DEBUG:urllib3.connectionpool:http://10.50.0.41:8080 "POST /graphql HTTP/1.1" 200 287
ERROR:root:Cannot add the relation, Stix-Domain-Object cannot be found.
ERROR:root:{'name': 'FunctionalError', 'message': 'Cannot add the relation, Stix-Domain-Object cannot be found.'}`
Additional information
malpedia connector config:
connector-malpedia: image: opencti/connector-malpedia:latest environment: - OPENCTI_URL=http://xx.xx.xx.xx:8080 - OPENCTI_TOKEN=1fba4df1-xxxx-3f3520341ac9 - CONNECTOR_ID=malpedia_connector - CONNECTOR_TYPE=EXTERNAL_IMPORT - CONNECTOR_NAME=Malpedia - CONNECTOR_SCOPE=malpedia - CONNECTOR_CONFIDENCE_LEVEL=30 # From 0 (Unknown) to 100 (Fully trusted) - CONNECTOR_UPDATE_EXISTING_DATA=false - CONNECTOR_LOG_LEVEL=debug - MALPEDIA_AUTH_KEY= # Empty key only fetches TLP:WHITE information - MALPEDIA_INTERVAL_SEC=86400 # Run once every day - MALPEDIA_IMPORT_INTRUSION_SETS=true - MALPEDIA_IMPORT_YARA=false - MALPEDIA_CREATE_INDICATORS=true - MALPEDIA_CREATE_OBSERVABLES=true restart: always
Thanks for reporting - I'll look into this. NOTE: The malpedia team is currently working on a new STIX2 export that might render a lot of the current connector obsolete.
I'm debugging locally and I can see that the Malware Stix-Domain-Object (SDO) is being added successfully to OpenCTI but it seems like the immediate subsequent request to add an external reference to it fails due to the Malware SDO object not being found. I tried running the same GraphQL mutation that the connector is using with the same input and was successfully able to add the external reference. This leads me to believe that there may be a race condition on the server side however unlikely that may seem? With that said, the connector probably needs to be rewritten as it ingests data into OpenCTI using the API rather than workers as recommended in the guide. But any pointers you can provide to resolve this would be helpful as I don't see anything obviously wrong w/ the connector code. Thanks!
Hello, an update, still seeing errors on Malpedia :
INFO:root:Listing Malwares with filters [{"key": "name", "values": ["Sparksrv"]}].
INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["Sparksrv"]}].
INFO:root:Tag 'Sparksrv' does not reference malware
INFO:root:Listing Malwares with filters [{"key": "name", "values": ["win.sparksrv"]}].
INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["win.sparksrv"]}].
INFO:root:Tag 'win.sparksrv' does not reference malware
INFO:root:Reading Marking-Definition {marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9}.
INFO:root:Creating Malware {Sparksrv}.
INFO:root:Creating External Reference {Malpedia}.
INFO:root:Adding External-Reference {198141f9-952e-4a23-a9b4-662bca4fc832} to Stix-Domain-Object {c9427149-3377-461c-a90c-2047a38dce10}
ERROR:root:Cannot add the relation, Stix-Domain-Object cannot be found.
ERROR:root:{'name': 'FunctionalError', 'message': 'Cannot add the relation, Stix-Domain-Object cannot be found.'}
I have the same message with opencti 5.2.3 :
août 22 11:44:11 : INFO:root:starting Malpedia connector... août 22 11:44:13 : INFO:root:current Malpedia version: 15555 août 22 11:44:13 : INFO:root:loaded state: {} août 22 11:44:13 : INFO:root:running importers août 22 11:44:13 : INFO:root:running Knowledge importer with state: {} août 22 11:44:13 : INFO:root:Reading Marking-Definition {marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9}. août 22 11:44:13 : INFO:root:Reading Marking-Definition {marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da}. août 22 11:44:13 : INFO:root:Reading Marking-Definition {marking-definition--f88d31f6-486f-44da-b317-01333bde0b82}. août 22 11:44:14 : INFO:root:Reading Marking-Definition {marking-definition--5e57c739-391a-4eb3-b6be-7d15ca92d5ed}. août 22 11:44:14 : INFO:root:Processing malware family: aix.fastcash août 22 11:44:14 : INFO:root:Processing malware family: aix.fastcash août 22 11:44:14 : INFO:root:Listing Malwares with filters [{"key": "name", "values": ["FastCash"]}]. août 22 11:44:14 : INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["FastCash"]}]. août 22 11:44:15 : INFO:root:Tag 'FastCash' does not reference malware août 22 11:44:15 : INFO:root:Listing Malwares with filters [{"key": "name", "values": ["aix.fastcash"]}]. août 22 11:44:15 : INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["aix.fastcash"]}]. août 22 11:44:15 : INFO:root:Tag 'aix.fastcash' does not reference malware août 22 11:44:15 : INFO:root:Reading Marking-Definition {marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9}. août 22 11:44:15 : INFO:root:Creating Malware {FastCash}. août 22 11:44:17 : INFO:root:Creating External Reference {Malpedia}. août 22 11:44:17 : INFO:root:Adding External-Reference {80940ebf-ec45-4ec7-ad1a-351822c3cf69} to Stix-Domain-Object {43f6d152-e191-4531-beae-e6ba0b0effa5} août 22 11:44:17 : ERROR:root:Cannot add the relation, Stix-Domain-Object cannot be found. août 22 11:44:17 : ERROR:root:{'name': 'FunctionalError', 'message': 'Cannot add the relation, Stix-Domain-Object cannot be found.'}
Still seeing same issue (anyway to get past this running v5.3.15?):
INFO:root:Creating Identity {Malpedia}. INFO:root:starting Malpedia connector... INFO:root:current Malpedia version: 16119 INFO:root:loaded state: {} INFO:root:running importers INFO:root:running Knowledge importer with state: {} INFO:root:Reading Marking-Definition {marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9}. INFO:root:Reading Marking-Definition {marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da}. INFO:root:Reading Marking-Definition {marking-definition--f88d31f6-486f-44da-b317-01333bde0b82}. INFO:root:Reading Marking-Definition {marking-definition--5e57c739-391a-4eb3-b6be-7d15ca92d5ed}. INFO:root:Processing malware family: aix.fastcash INFO:root:Processing malware family: aix.fastcash INFO:root:Listing Malwares with filters [{"key": "name", "values": ["FastCash"]}]. INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["FastCash"]}]. INFO:root:Tag 'FastCash' does not reference malware INFO:root:Listing Malwares with filters [{"key": "name", "values": ["aix.fastcash"]}]. INFO:root:Listing Malwares with filters [{"key": "aliases", "values": ["aix.fastcash"]}]. INFO:root:Tag 'aix.fastcash' does not reference malware INFO:root:Reading Marking-Definition {marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9}. INFO:root:Creating Malware {FastCash}. INFO:root:Creating External Reference {Malpedia}. INFO:root:Adding External-Reference {ada1969f-477e-4c47-9b4e-a2e1cbc87c91} to Stix-Domain-Object {d1ca16b4-7653-4f19-b6e8-01ff523df348} ERROR:root:Cannot add the relation, Stix-Domain-Object cannot be found. ERROR:root:{'name': 'FunctionalError', 'message': 'Cannot add the relation, Stix-Domain-Object cannot be found.'}
