signoz
signoz copied to clipboard
collector processors generated for Log pipelines should not be async
Logs pipelines in the UI get translated to logstransform processors right now. logstransform processors are async and do their own internal batching. This makes the entire collector pipeline async - consumeLogs at the receiver returns before it is guaranteed that the exporter has finished exporting logs. Having sync collector pipelines makes it simpler to provide persistence guarantees for logs collected by receivers.
We would appreciate it if you could provide us with more info about this issue/pr!