Feature Request: Pluggable Collector Architecture for Custom Data Ingestion

[officiallyutso]

Motivation

In enterprise and multi-cloud environments, there is often a need to enrich flow data or forward telemetry to custom systems such as internal threat intelligence platforms, legacy SIEMs, or proprietary analytics engines. Currently, extending Retina’s collection or enrichment logic requires modifying the core codebase, which introduces maintenance challenges and limits safe integration paths.

Proposal

Introduce a pluggable collector interface that allows external developers to define, load, and run custom collectors or enrichers without modifying Retina's core.

• Example interface in Go:

type CollectorPlugin interface {
    Init(config map[string]string) error
    Start(ctx context.Context, emitter EventEmitter) error
    Stop() error
    Metadata() PluginMetadata
}

• Collectors could be configured dynamically:

customCollectors:
  - name: "custom-enricher"
    type: "native"
    path: "/opt/retina/plugins/custom_enricher.so"

Optional support for Wasm-based plugins could provide security and isolation for untrusted modules.

Benefits

• Enables domain-specific enrichment and data forwarding pipelines

• Reduces friction in adopting Retina across varied environments

• Decouples internal logic from enterprise-specific extensions

• Encourages community contributions of specialized plugins

Considerations

• Plugin lifecycle and failure isolation

• Versioning and compatibility guarantees

• Optional sandboxing for untrusted code (e.g., Wasm or subprocess isolation)

If this direction aligns with Retina’s roadmap, I would be happy to contribute to an initial draft or prototype implementation.

[vakalapa]

The motivation is clear: enriching flow data or forwarding telemetry to internal systems like threat intel platforms or legacy SIEMs is a common requirement. Today, doing this in Retina requires modifying core logic, which increases maintenance overhead and limits safe extensibility.

We are happy to take contributions on this, also it would be ideal to have some example collectors to start off with as well.