kubernetes-event-exporter icon indicating copy to clipboard operation
kubernetes-event-exporter copied to clipboard

Checkpointing for Event loss and deduplication

Open akifkhan01 opened this issue 4 years ago • 1 comments

This component is using k8s.io/client-go/informers to create a watch on K8s-Event object, this informer reads resource ( K8s-Event ) from the source which can cause duplication of data in sink in case of event-exporter going down, to avoid duplication and no loss of Event data, we need checkpointing/bookmark in the application.

Is it handled explicitly by k8s.io/client-go/informers ? Did we think about these failure scenarios, if yes, what was the conclusion ?

@mustafaakin @brianterry

akifkhan01 avatar Jan 29 '21 10:01 akifkhan01

These can be added, but it won't be a simple product then, that we need to have that state with also the leader election. Feel free to work out details if you are willing to contribute. During the original design, we accepted that risk.

mustafaakin-atl avatar Jun 16 '21 21:06 mustafaakin-atl