spring-projects
Per meterid metrics distribution.percentiles-histogram expose incorrectly active metrics
When using explicit serviceLevelObjectiveBoundaries in prometheus metrics like
management.prometheus.metrics.export.enabled=true
management.metrics.distribution.slo.http=30,50,100,150,200,300,500,700,1s,1500,2s,3s,4s,5s,7s,10s,15s,20s,30s
Then only "http_server_requests_seconds_bucket" and "http_client_requests_seconds_bucket" are correctly created.
http_server_requests_seconds_bucket{app="test-v1-v",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/test",le="0.03"} 0
http_server_requests_seconds_bucket{app="test-v1-v",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/test",le="0.05"} 0
However when using "percentiles-histogram" the output contains also histogram 28 buckets for "active" metrics (from 120ms to 2h) like "http_server_requests_active_seconds_bucket" and "http_client_requests_active_seconds_bucket"
http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="120.0"} 1
http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="137.438953471"} 1
.......
http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="7200.0"} 1
http_server_requests_active_seconds_bucket{app="test-v1",exception="none",method="GET",outcome="SUCCESS",springBoot="3.5.8",status="200",uri="UNKNOWN",le="+Inf"} 1
Trying to exclude "active" is not possible. Limiting buckets to metric id "http.server.requests" still include active "http_server_requests_active_seconds_bucket" buckets.
management.metrics.distribution.percentiles-histogram.http.server.requests=true
management.metrics.distribution.minimum-expected-value.http=40
management.metrics.distribution.maximum-expected-value.http=30s
And not possible to add "seconds", because "seconds" is not metrics id, it is added automatically for legacy histograms
management.metrics.distribution.percentiles-histogram.http.server.requests.seconds=true # not working
management.metrics.distribution.minimum-expected-value.http=40
management.metrics.distribution.maximum-expected-value.http=30s
Now slo and percentiles-histogram behave inconsistently.
It should be possible to control if active metrics are included into the buckets. Or have possibility to exclude active metrics.
Tested in: SpringBoot version: 3.5.8 Windows 11 Java 17
The properties that you're configuring are used by Spring Boot's PropertiesMeterFilter where it applies them in its configure(Id, DistributionStatisticConfig) method. I don't believe that DistributionStatisticConfig gives us any control over what is included in the buckets. In other words, I think the functionality that you would like to see will require a change in Micrometer. If I have missed something and you can configure things as you'd like using your own MeterFilter implementation, please share an example of that and we can take another look and see how that might translate to property-based configuration. Otherwise, please open a Micrometer issue for their consideration.
If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.
