The connector opencti did not work

[highkay]

What happened

The connector opencti failed.

Environment

1. OS: docker
2. IntelOwl version: 3.3.1

What did you expect to happen

connector successed.

How to reproduce your issue

I have config the connector. all (3 options) verified and the check button is ok.

the opencti url I config is http://x.x.x.x:x/

1. start a scan a observable with a connector
2. all analyzer successed
3. the connector start and return an error

Error messages and logs

intelowl_celery_worker_default | [2022-03-09 08:10:20,925: INFO/ForkPoolWorker-18] [REPORT] <Job: Job(#11, "45.61.137.231")>, status:running, reports:{'all': 3, 'failed': 0, 'pending': 0, 'running': 0, 'success': 3, 'killed': 0}
intelowl_celery_worker_default | [2022-03-09 08:10:20,967: INFO/MainProcess] Task start_connectors[6ecda87d-a495-4f4b-88e1-681dbc521540] received
intelowl_celery_worker_default | [2022-03-09 08:10:20,986: INFO/ForkPoolWorker-18] Task post_all_analyzers_finished[72b2f236-bbce-41b2-aa42-566d467de1f5] succeeded in 0.14682591706514359s: None
intelowl_celery_worker_default | [2022-03-09 08:10:21,364: INFO/MainProcess] Task run_connector[0b59e458-c34e-498b-a786-7084615cd7d8] received
intelowl_celery_worker_default | [2022-03-09 08:10:21,403: INFO/ForkPoolWorker-7] Task start_connectors[6ecda87d-a495-4f4b-88e1-681dbc521540] succeeded in 0.07437220960855484s: None
intelowl_celery_worker_default | [2022-03-09 08:10:22,213: INFO/ForkPoolWorker-8] STARTED connector: (OpenCTI, job: #11)
intelowl_celery_worker_default | Traceback (most recent call last):
intelowl_celery_worker_default |   File "/opt/deploy/intel_owl/api_app/core/classes.py", line 123, in start
intelowl_celery_worker_default |     _result = self.run()
intelowl_celery_worker_default |   File "/opt/deploy/intel_owl/api_app/connectors_manager/connectors/opencti.py", line 109, in run
intelowl_celery_worker_default |     self.opencti_instance = pycti.OpenCTIApiClient(
intelowl_celery_worker_default |   File "/usr/local/lib/python3.9/site-packages/pycti/api/opencti_api_client.py", line 179, in __init__
intelowl_celery_worker_default |     raise ValueError(
intelowl_celery_worker_default | ValueError: OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...
intelowl_celery_worker_default | [2022-03-09 08:10:22,276: ERROR/ForkPoolWorker-8] (OpenCTI, job: #11). Unexpected error: 'OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...'
intelowl_celery_worker_default | Traceback (most recent call last):
intelowl_celery_worker_default |   File "/opt/deploy/intel_owl/api_app/core/classes.py", line 123, in start
intelowl_celery_worker_default |     _result = self.run()
intelowl_celery_worker_default |   File "/opt/deploy/intel_owl/api_app/connectors_manager/connectors/opencti.py", line 109, in run
intelowl_celery_worker_default |     self.opencti_instance = pycti.OpenCTIApiClient(
intelowl_celery_worker_default |   File "/usr/local/lib/python3.9/site-packages/pycti/api/opencti_api_client.py", line 179, in __init__
intelowl_celery_worker_default |     raise ValueError(
intelowl_celery_worker_default | ValueError: OpenCTI API is not reachable. Waiting for OpenCTI API to start or check your configuration...
intelowl_celery_worker_default | [2022-03-09 08:10:22,278: INFO/ForkPoolWorker-8] FINISHED connector: (OpenCTI, job: #11)

[mlodic]

hey, thanks for reporting this.

The "Health Check" performs a simple "HEAD" HTTP request to the URL that you configured in url_key_name parameter in the connector_config.json file. (the URL of your OpenCTI instance). This is to understand if there is a reachable instance. (so it does not tests whether your API token is correct or not, this test is the same for all the connectors). If that worked with status "healthy" this means that IntelOwl can reach OpenCTI.

On the contrary, the error you got is from the pycti library and it performs another test that is application specific. In particular, it wont' work if the API token is not correct. Could you please check that value? The error should be there

[highkay]

I use the api key in opencti setting page. It is not valid?Or I should use some key in other page?

[mlodic]

It should be correct. Is that correctly inserted in the CONNECTOR_OPENCTI_KEY variable in the env_file_app file?

If yes, I don't know, it could some compatibility problems between different versions of OpenCTI. Should be investigated.

[highkay]

I am sure the key is setting correctly.

[mlodic]

So well I guess we need to make some manual tests to try to reproduce your error and find where is the problem. I see when I can have some spare time to do that.

I'll insert other questions that can help in debugging this in this thread

Is it the first time you tried that connector? Did it work before?

[highkay]

So well I guess we need to make some manual tests to try to reproduce your error and find where is the problem. I see when I can have some spare time to do that.

I'll insert other questions that can help in debugging this in this thread

Is it the first time you tried that connector? Did it work before?

Yes, it is my first time to use connector.

[sp35]

@mlodic Let's override the health check to use opencti_client.health_check; found it in the init logic here

[mlodic]

great idea! I found that check but did not think about the overwrite!

[mlodic]

I tried very fast to make the overwrite but noticed that the health_check class of Connector is a classmethod. This complicates and makes difficult the overwrite because we would need data from an instance of the Connector, while the health_check works without an instance.

I think that it could make sense to create "custom" and more advanced "health checks" for the connectors based on the specific technlogies (like the one you mentioned but for every connector). I can open a new issue.

Here we just need to understand if there are some compatibility issues.

[intbjw]

I also have this issue,

How do I configure it to work? I can't find anything in the official documentation about it

[mlodic]

Are you sure that you have correctly configured the URL? As I said before, The "Health Check" performs a simple "HEAD" HTTP request to the URL that you configured in url_key_name parameter in the connector_config.json file.. So if the instance is up and you configured your OS and network correctly, this should work.

Have you manually tried an analysis to see if the output goes correctly to OpenCTI?

[mlodic]

see #1730 to solve the problem