spring-kafka
spring-kafka copied to clipboard
spring-projects
Enable KafkaStreamsMicrometerListener based on configuration
Spring Kafka provides KafkaStreamsMicrometerListener, which currently has to be added to the StreamsBuilderFactoryBean programmatically. There should be a configuration/property that supports adding a default listener automatically. This is similar to spring.kafka.listener.observation-enabled and spring.kafka.template.observation-enabled. Perhaps, something like spring.kafka.streams.observation-enabled? For now, it only adds the metrics; when the feature is ready, it can also auto-configure traces (#2635 and https://github.com/micrometer-metrics/micrometer/issues/3713) like it happens for KafkaTemplate and KafkaListener.
@singhbaljit As of now, there hasn't been any progress on adding observability natively from Kafka Streams. I don't believe we can add a property like observation-enabled and expect it to work since the underlying library does not support observability yet. Maybe as part of Spring Kafka (3.3.0), this is something we can come back and visit again to assess the situation.
So, does KafkaStreamsMicrometerListener not do anything? I'm trying to make sense of the documentation.
I does only KafkaStreamsMetrics for the MeterRegistry.
I understand that observability covers metrics as well, but that might confuse users since there is no tracing for Kafka Streams yet.
Either way, what you are asking for might belong to Spring Boot and its configuration properties.
Whatever Spring for Apache Kafka provides simply can be configured as you mentioned: via setters on the StreamsBuilderFactoryBean.
If it is best to have this issue in the Spring Boot project, that is alright with me (please move it). All I'm really looking for is to have the "default" observability components auto-configured as part of the auto-configured StreamsBuilderFactoryBean, whichever observability components are currently supported. I'm assuming not everyone will want this enabled by default, so a configuration property to switch this on/off makes sense.
Calling the property observation-enabled will give the impression that Kafka Streams provides observation (as Artem mentioned), regardless of whether it is provided from Boot or Spring Kafka. Until the Kafka Streams observability story gets some progress, I think it is better to do it at the application level.