connectors icon indicating copy to clipboard operation
connectors copied to clipboard

[stream replication] first version of connectors

Open axelfahy opened this issue 4 months ago • 0 comments

This PR adds two new connectors, a stream/stream-exporter and a external-import/stream-importer. The goal is to be able to synchronize two OpenCTI instances that don't have access to each other.

The stream-exporter reads a live-stream and save all events on minio. The stream-importer reads the files from minio and send them to rabbitmq to be inserted in OpenCTI by the workers.

Proposed changes

  • Add a stream/stream-exporter connector that reads events from a live stream and save them to minio (in bulk).
  • Add a external-importer/stream-importer connector that reads files from minio (containing the events of the live stream) and send them to rabbitmq to be imported in OpenCTI.

Related issues

  • N/A

Checklist

  • [x] I consider the submitted work as finished
  • [x] I tested the code for its functionality using different use cases
  • [x] I added/update the relevant documentation (either on github or on notion)
  • [x] Where necessary I refactored code to improve the overall quality

Further comments

axelfahy avatar Sep 30 '24 12:09 axelfahy