connectors icon indicating copy to clipboard operation
connectors copied to clipboard

Stream Connector goes to "Inactive" status, does not "StreamAlive"

Open timebotdon opened this issue 1 year ago • 2 comments

Description

We currently have installed 2 Qradar Stream connectors. Alienvault and Mandiant, respectively. One of them goes to "Inactive" status when there are no objects being sent to the QRadar reference sets. This happens every 5 minutes or so whenever I attempt to restart the connector. It appears that the connector halts all operations, including "StreamAlive", whenever this happens.

Environment

  1. OS (where OpenCTI server runs): Ubuntu 22.04
  2. OpenCTI version: OpenCTI 6.0.9 Community Edition
  3. OpenCTI client: frontend / python-client
  4. Other environment details: N/A

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. configure loglevel to "debug" in config.yml
  2. run connector: python3 qradar.py
  3. When there are no objects streamed to Qradar, it stops responding. No messages are outputted to the log.

Expected Output

The connector runs normally, "StreamAlive" is running.

...
{"timestamp": "2024-05-15T10:01:58.625648Z", "level": "DEBUG", "name": "QRadar Stream (Mandiant)", "message": "processing message with id 560d73c5-76b3-4abd-b99f-ab16f033e8aa"}
{"timestamp": "2024-05-15T10:01:59.087205Z", "level": "DEBUG", "name": "QRadar Stream (Mandiant)", "message": "StreamAlive running"}
{"timestamp": "2024-05-15T10:02:03.439300Z", "level": "DEBUG", "name": "QRadar Stream (Mandiant)", "message": "PingAlive running."}
{"timestamp": "2024-05-15T10:02:04.092535Z", "level": "DEBUG", "name": "QRadar Stream (Mandiant)", "message": "StreamAlive running"}

Actual Output

The connector does not keep the stream alive. "StreamAlive" messages no longer appear on the console.

...
{"timestamp": "2024-05-15T09:02:52.681940Z", "level": "DEBUG", "name": "QRadar Stream (Alienvault)", "message": "reference_set item with id 7a6f4616-5728-4674-8cdc-0dd5a55c0eeb created"}
{"timestamp": "2024-05-15T09:02:52.682023Z", "level": "INFO", "name": "QRadar Stream (Alienvault)", "message": "created IOC: https://redacted"}
{"timestamp": "2024-05-15T09:02:52.710937Z", "level": "DEBUG", "name": "QRadar Stream (Alienvault)", "message": "processing message with id 329ef266-d195-4b16-a09e-a8b901af3cc5"}
{"timestamp": "2024-05-15T09:02:52.712143Z", "level": "DEBUG", "name": "QRadar Stream (Alienvault)", "message": "processing message with id 1f9eb69c-90ff-4e66-a713-4f5867d9c588"}

Additional information

I have noticed that no new objects are being streamed to QRadar when there are new IOC objects being ingested by OpenCTI. It seems like the connector has shutdown completely, even though the python process is still running.

timebotdon avatar May 15 '24 10:05 timebotdon

@timebotdon If I understand well the issue only happens for QRadar? Could it be linked to the deprecated API?

nino-filigran avatar May 16 '24 07:05 nino-filigran

@timebotdon If I understand well the issue only happens for QRadar? Could it be linked to the deprecated API?

No, the problem persists even when the deprecated API is updated. This happens to just 1 stream connector. The other qradar connector is working. As you can see, the Mandiant one appears to be streaming normally, while the other, does not.

timebotdon avatar May 16 '24 15:05 timebotdon