Spring Boot instrumentation starter disables default SimpleMeterRegistry

[ViliusS]

Describe the bug If instrumentation-spring-boot-starter is added to Spring Boot application it breaks /actuator/metrics endpoint. When we query /actuator/metrics endpoint all of the metrics are still available in the names list, however when we query the metric itself, for example /actuator/metrics/application.started.time it shows just an empty value.

I tracked it down to the usage of Micrometer shim which for some reason replaces SimpleMeterRegistry with OpenTelemetryMeterRegistry.

Steps to reproduce

1. Start empty Spring Boot project with Spring Boot Actuator dependency and management.endpoints.web.exposure.include=* property.
2. Check /actuator/metrics/application.started.time endpoint. You should see correct metric value.
3. Import opentelemetry-spring-boot-starter in POM file.
4. Check /actuator/metrics/application.started.time endpoint again. The metric is empty at this point.

What did you expect to see? Spring Boot instrumentation should not disable in-memory SimpleMeterRegistry

What version are you using? OpenTelemetry instrumentation 1.20.2-alpha Spring Boot 2.7.4

Additional context More information and debugging flow is available at https://github.com/spring-projects/spring-boot/issues/33449

[zeitlinger]

What happens

By default, Spring Boot uses the SimpleMeterRegistry, which does show the value

curl -s http://localhost:8080/actuator/metrics/application.started.time
{"name":"application.started.time","description":"Time taken to start the application","baseUnit":"seconds","measurements":[{"statistic":"VALUE","value":0.991}],"availableTags":[{"tag":"main.application.class","values":["com.grafana.demo.DemoApplication"]}]}

If any metrics registry is registered, then the simple metrics registry will not be used any longer - it's a fallback. (You could argue that this is a bug in spring boot itself - but that would have to be filed in spring boot.)

You still get data on the endpoint for the simple metrics registry (the output below) - but the data is served from the first registered meter registry.

curl -s http://localhost:8080/actuator/metrics/application.started.time
{"name":"application.started.time","description":"Time taken to start the application","baseUnit":"seconds","measurements":[{"statistic":"VALUE","value":"NaN"}],"availableTags":[{"tag":"main.application.class","values":["com.grafana.demo.DemoApplication"]}]}

So it depends on the capabilities of that registry. The Micrometer bridge that is used in our case does not support reading measurements - which is used for the command above (but otherwise not necessary to export data using OTLP).

Workarounds

Fortunately, there's an easy workaround: Add the prometheus registry as shown below. It supports the required functionality and happes to be registered before the micrometer bridge of this starter.

runtimeOnly 'io.micrometer:micrometer-registry-prometheus'

If you absolutely cannot add prometheus starter, you can add a modified simple meter registry that does not back off - and that is registered first:

@AutoConfiguration(before = {CompositeMeterRegistryAutoConfiguration.class, MicrometerBridgeAutoConfiguration.class}, after = MetricsAutoConfiguration.class)
@ConditionalOnBean(Clock.class)
@EnableConfigurationProperties(SimpleProperties.class)
//@ConditionalOnMissingBean(MeterRegistry.class)
@ConditionalOnEnabledMetricsExport("simple")
public class OtelSimpleMetricsExportAutoConfiguration {

	@Bean
	public SimpleMeterRegistry simpleMeterRegistry(SimpleConfig config, Clock clock) {
		return new SimpleMeterRegistry(config, clock);
	}

	@Bean
	@ConditionalOnMissingBean
	public SimpleConfig simpleConfig(SimpleProperties simpleProperties) {
		return new SimplePropertiesConfigAdapter(simpleProperties);
	}

}

[ViliusS]

There is another workaround I have mentioned in one of the linked issues. You can set otel.springboot.micrometer.enabled=false to make simple meter registry work again. I suppose it disables Micrometer bridge and makes Spring Boot think none of the registries are present.

[zeitlinger]

There is another workaround I have mentioned in one of the linked issues. You can set otel.springboot.micrometer.enabled=false to make simple meter registry work again. I suppose it disables Micrometer bridge and makes Spring Boot think none of the registries are present.

Thanks - I totally forgot this option.

BTW, since 2.0.0, otel.springboot.micrometer.enabled=false is the default option, because most of the time, you don't need it, because the metrics produced by micrometer are very similar to the metrics that are produced by the instrumentations included in this starter.

[oldium]

Just a note, the property name since OTEL Java Instrumentation v1.29.0 is otel.instrumentation.micrometer.enabled (i.e. env name OTEL_INSTRUMENTATION_MICROMETER_ENABLED)