openmeter
openmeter copied to clipboard
Process Messages in DQL (Dead Letter Queue)
When Kafka Connect ClickHouse gets a malformed message, it sends the entire batch to DQL. It means the DQL topic can have bad and good messages mixed. Ideally, OpenMeter would have a secondary consumer for DQL, which runs validation against those messages and tries to re-process the ones that it finds the corresponding meter and matches the validator.
Related to: https://github.com/openmeterio/openmeter/pull/225