opentelemetry-collector icon indicating copy to clipboard operation
opentelemetry-collector copied to clipboard

[chore] [exporterhelper] Integrate capacity limiting into the communication channel

Open dmitryax opened this issue 1 year ago • 4 comments
trafficstars

Integrate capacity limiting into internal channels used by both memory and persistent queues. Otherwise, with the independent capacity limiter, it's hard to ensure that queue size is always accurate going forward.

Benchmarks before:

goos: darwin
goarch: arm64
Benchmark_QueueUsage_1000_requests-10      	    3252	    325010 ns/op	  246059 B/op	      10 allocs/op
Benchmark_QueueUsage_100000_requests-10    	      39	  29811116 ns/op	24002870 B/op	      10 allocs/op
Benchmark_QueueUsage_10000_items-10        	    3404	    349753 ns/op	  246052 B/op	      10 allocs/op
Benchmark_QueueUsage_1M_items-10           	      40	  29415583 ns/op	24002858 B/op	      10 allocs/op
BenchmarkPersistentQueue_TraceSpans
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_1-10         	  338180	      3836 ns/op	    2851 B/op	      78 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_10-10        	   81369	     15822 ns/op	   14598 B/op	     289 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_10_#spansPerTrace:_10-10       	   13066	     90155 ns/op	  130087 B/op	    2417 allocs/op

Benchmarks after:

Benchmark_QueueUsage_1000_requests-10      	    4210	    278175 ns/op	  246055 B/op	      10 allocs/op
Benchmark_QueueUsage_100000_requests-10    	      42	  25835945 ns/op	24002968 B/op	      10 allocs/op
Benchmark_QueueUsage_10000_items-10        	    4376	    279571 ns/op	  246056 B/op	      10 allocs/op
Benchmark_QueueUsage_1M_items-10           	      42	  26483907 ns/op	24002995 B/op	      10 allocs/op
BenchmarkPersistentQueue_TraceSpans
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_1-10         	  328268	      4251 ns/op	    2854 B/op	      78 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_1_#spansPerTrace:_10-10        	  101683	     12238 ns/op	   14582 B/op	     289 allocs/op
BenchmarkPersistentQueue_TraceSpans/#traces:_10_#spansPerTrace:_10-10       	   13382	     86464 ns/op	  130154 B/op	    2417 allocs/op

dmitryax avatar Jan 06 '24 03:01 dmitryax

Codecov Report

Attention: Patch coverage is 88.18182% with 13 lines in your changes are missing coverage. Please review.

Project coverage is 91.86%. Comparing base (67d3718) to head (999718d).

Files Patch % Lines
exporter/internal/queue/persistent_queue.go 77.96% 10 Missing and 3 partials :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9232      +/-   ##
==========================================
- Coverage   91.87%   91.86%   -0.02%     
==========================================
  Files         360      361       +1     
  Lines       16717    16722       +5     
==========================================
+ Hits        15359    15361       +2     
- Misses       1020     1024       +4     
+ Partials      338      337       -1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Jan 06 '24 03:01 codecov[bot]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

github-actions[bot] avatar Mar 09 '24 03:03 github-actions[bot]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

github-actions[bot] avatar Mar 26 '24 03:03 github-actions[bot]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

github-actions[bot] avatar Apr 11 '24 03:04 github-actions[bot]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

github-actions[bot] avatar May 01 '24 03:05 github-actions[bot]