opentelemetry-go
opentelemetry-go copied to clipboard
open-telemetry
sdk/log: Redesign BatchProcessor
Maybe the goroutine responsible for actually exporting the log records should be also responsible for getting/dequeuing the log records from the queue instead of doing it here?
My idea was that here we only signal that we want to have the exporter goroutine (processRecordsBatches) to try to export. The exporter goroutine could dequeue by itself. I do not think we need a sync.Pool for batches as we have only one exporting goroutine.
The idea is also to get rid of something like len(b.recordsBatches) == cap(b.recordsBatches) (or len(e.input) != cap(e.input)) which looks like a code smell.
Currently, the "poll goroutine" can consume almost a whole CPU thread when there is "big traffic" and bufferExporter.Ready returns false over and over (which results in no-op for loop) .
Originally posted by @pellared in https://github.com/open-telemetry/opentelemetry-go/pull/6673#discussion_r2097675356
Created for tracking and assigning myself as "sponsor".
@wmdanor, feel free to continue working on it if you are interested. However, giving my laconic description and unclear outcome, I totally understand if you do not want to abandon it.
Conversation that may be helpful: https://github.com/open-telemetry/opentelemetry-go/pull/6673#discussion_r2097675356
