signoz icon indicating copy to clipboard operation
signoz copied to clipboard

collector processors generated for Log pipelines should not be async

Open raj-k-singh opened this issue 2 years ago • 1 comments

Logs pipelines in the UI get translated to logstransform processors right now. logstransform processors are async and do their own internal batching. This makes the entire collector pipeline async - consumeLogs at the receiver returns before it is guaranteed that the exporter has finished exporting logs. Having sync collector pipelines makes it simpler to provide persistence guarantees for logs collected by receivers.

raj-k-singh avatar Jan 28 '24 13:01 raj-k-singh

We would appreciate it if you could provide us with more info about this issue/pr!

request-info[bot] avatar Jan 28 '24 13:01 request-info[bot]