opentelemetry-python
opentelemetry-python copied to clipboard
open-telemetry
Measure number of dropped spans in `BatchSpanProcessor`
Is your feature request related to a problem?
We would like to measure the number of dropped spans with the BatchSpanProcessor, simple True / False flag is not sufficient, as some (low) numbers of drops might not indicate anything bad.
Describe the solution you'd like
Having a simple way to add a meter without having to override or wrap on_exit to see how many spans were dropped because of a full queue. Ideally, a call to a callback when spans are dropped due to a full queue.
Describe alternatives you've considered
Re-implementing the on_exit of BatchSpanProcessor with added on_drop callback with a meter or adding a meter directly there. Eventually wrapping the existing method by checking the _spans_dropped flag and changing it back to false after incrementing a meter. Both methods might interfere with future changes, the latter would increase the log rate.
Are you trying to access the counts programmatically or just monitor the queue? In the spirit of OTel, we should consider adding the metric instrumentation directly to the BatchSpanProcessor code.
I am just trying to monitor it, so doing it that way would also solve the issue for me and I don't need to have the number anywhere else.
I have submitted a PR for this, Can someone please review and suggest improvements?
Was having a look at this issue and noticed that when adding spans to the BatchSpanProcessor's queue, its size is never greater than the max_export_batch_size. BatchSpanProcessor validates that the max_export_batch_size is always <= max_queue_size (see _validate_arguments function that throws an error when max_export_batch_size > max_queue_size). This means that it is not possible to have any dropped spans. Please correct me if I am wrong.
