connectors
connectors copied to clipboard
[stream replication] first version of connectors
This PR adds two new connectors, a stream/stream-exporter
and a external-import/stream-importer
. The goal is to be able to synchronize two OpenCTI instances that don't have access to each other.
The stream-exporter
reads a live-stream and save all events on minio.
The stream-importer
reads the files from minio and send them to rabbitmq to be inserted in OpenCTI by the workers.
Proposed changes
- Add a
stream/stream-exporter
connector that reads events from a live stream and save them to minio (in bulk). - Add a
external-importer/stream-importer
connector that reads files from minio (containing the events of the live stream) and send them to rabbitmq to be imported in OpenCTI.
Related issues
- N/A
Checklist
- [x] I consider the submitted work as finished
- [x] I tested the code for its functionality using different use cases
- [x] I added/update the relevant documentation (either on github or on notion)
- [x] Where necessary I refactored code to improve the overall quality