MemoryLimiter extension as an http/grpc middleware
Description
This PR augments the memorylimiter extension to be an HTTP/GRPC middleware. The extension can be used as an HTTP/GRPC middleware once this is merged. Example:
receivers:
otlp:
protocols:
grpc:
middlewares:
- id: memory_limiter
http:
middlewares:
- id: memory_limiter
extensions:
memory_limiter:
check_interval: 1s
limit_percentage: 1
spike_limit_percentage: 0.05
..........
Link to tracking issue
Fixes #9591 Fixes #14081
Testing
Setup a minimal collector to test that the middleware is indeed ratelimiting while the memory usage is closer to limit.
receivers:
otlp:
protocols:
grpc:
middlewares:
- id: memory_limiter
http:
middlewares:
- id: memory_limiter
processors:
exporters:
debug:
verbosity: detailed
sampling_initial: 5
sampling_thereafter: 200
use_internal_logger: false
extensions:
memory_limiter:
check_interval: 1s
limit_percentage: 1 # this is the minimum possible, since it's an integer.
spike_limit_percentage: 0.05
service:
extensions:
- memory_limiter
pipelines:
metrics:
receivers: [otlp]
processors: []
exporters: [debug]
traces:
receivers: [otlp]
processors: []
exporters: [debug]
logs:
receivers: [otlp]
processors: []
exporters: [debug]
telemetry:
logs:
level: debug
development: false
disable_caller: false
disable_stacktrace: false
Documentation
Updated the readme for memory limiter extension.
@shivanthzen Thank you. This looks like a good step forward. Can we add to the memorylimiter extension README.md with an example for use? Would you add a minimal test, at least to be sure that the extension implements the middleware interface? I am not sure what internal support memorylimiter has for testing, but it would be nice to have a small test for the function itself.
Codecov Report
:x: Patch coverage is 0% with 12 lines in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 92.21%. Comparing base (6341969) to head (52693d0).
:warning: Report is 20 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| extension/memorylimiterextension/memorylimiter.go | 0.00% | 12 Missing :warning: |
:x: Your patch check has failed because the patch coverage (0.00%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.
Additional details and impacted files
@@ Coverage Diff @@
## main #14071 +/- ##
==========================================
- Coverage 92.25% 92.21% -0.04%
==========================================
Files 658 658
Lines 41184 41196 +12
==========================================
- Hits 37993 37990 -3
- Misses 2184 2198 +14
- Partials 1007 1008 +1
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.