openmeter icon indicating copy to clipboard operation
openmeter copied to clipboard

Process Messages in DQL (Dead Letter Queue)

Open hekike opened this issue 10 months ago • 0 comments

When Kafka Connect ClickHouse gets a malformed message, it sends the entire batch to DQL. It means the DQL topic can have bad and good messages mixed. Ideally, OpenMeter would have a secondary consumer for DQL, which runs validation against those messages and tries to re-process the ones that it finds the corresponding meter and matches the validator.

Related to: https://github.com/openmeterio/openmeter/pull/225

hekike avatar Sep 01 '23 10:09 hekike