opentelemetry-collector
opentelemetry-collector copied to clipboard
(exporterhelper): Add concurrency sender to send batches concurrently
Description
This PR adds in a new component to the exporterhelper called the concurrency sender. This component is meant to limit the concurrency of RPC's independent from the queue sender's num_consumers
. This functionality is desirable because the queue_sender does not apply backpressure, so the concurrency_sender can limit requests based on a configured concurrency limit. Additionally, the batch_sender can generate multiple batches from a request that are exported sequentially, so the concurrency_sender provides a way to export these batches in parallel.
This PR also changes the requestSender
interface
type requestSender interface {
component.Component
send(context.Context, ...Request) error
setNextSender(nextSender requestSender)
}
This allows us to optionally send multiple requests, which will allow batch producers to receive any errors associated with any of their exported batches.
Link to tracking issue
Related to https://github.com/open-telemetry/opentelemetry-collector/issues/10368
Testing
Added unit tests and performed local testing to ensure concurrency works and errors are propagated back to batch producers