connectors icon indicating copy to clipboard operation
connectors copied to clipboard

feat: integrate relation extraction

Open SoniaBadene opened this issue 7 months ago • 1 comments

Proposed changes

  • Updated webservice call to use /extract_entities_relations endpoint with with_relations=true param.
  • Extended parsing logic to handle relationships from ML model response.
  • Tracked text-to-STIX ID mapping for relationship linking.
  • Added support for Identity types (Sector, Organization, Individual) and custom object Channel.
  • Created STIX relationships dynamically from predicted data when valid source/target IDs are found.
  • Preserved original fallback logic for known relationship rules (simplified the code).

TODO:

  • [ ] for perf, maybe avoid creating duplicate Relationship objects (rel_type, src_id, dst_id), so to have fewer instantiated Python objects, and if we need for later (text highlights) keep every textual mention by enriching the bundle with Note/Observed-Data ..

Related issues

  • #3986

SoniaBadene avatar May 20 '25 09:05 SoniaBadene

There is a typing issue related to self.file being implicitly defined as None in the __init__. We should initialize it there with self.file: dict[str, Any] | None = None

gregoirelafay avatar May 20 '25 12:05 gregoirelafay

During my review and tests, I encountered several configuration issues (entirely due to my own setup...), and unfortunately the container did not log any errors to stdout, only a generic "Terminated" message. I’d like to suggest, at least, updating main.py to display the full traceback upon failure, similar to the approach used in other connectors within the codebase

main.py:

# -*- coding: utf-8 -*-
"""OpenCTI ReportImporter connector main module."""

from reportimporter import ReportImporter

if __name__ == "__main__":
    import traceback
    try:
        connector = ReportImporter()
        connector.start()
    except Exception:
        traceback.print_exc()
        exit(1)

flavienSindou avatar Jul 30 '25 16:07 flavienSindou