inspectit-ocelot
inspectit-ocelot copied to clipboard
Wrong metrics referral causes inspectit not to expose any metric on prometheus metrics endpoint
The following example contains an error when refering to the metrics "pets". It references the view "pets/count" instead of the metric "pets" itself.
inspectit:
metrics:
definitions:
'[pets]':
type: LONG
unit: count
description: "The number of saves per pet type"
views:
'[pets/count]':
aggregation: COUNT
tags:
pet_type : true
instrumentation:
scopes:
's_save_pet':
type:
name: 'org.springframework.samples.petclinic.customers.web.PetService'
methods:
- name: save
matcher-mode: EQUALS_FULLY
rules:
'r_extract_pet_type':
scopes:
's_save_pet': true
metrics:
'[pets/count]':
value: 1
data-tags:
pet_type: 'extract-pet-type'
exit:
extract-pet-type:
action: 'a_extract_pet_type_from_save'
This leads to the following error message during startup of the instrumented java app:
021-11-17 12:41:54,155 ERROR 8635 --- [inspectIT] [ Thread-0] r.i.o.core.config.InspectitEnvironment : Error loading the configuration 'inspectit'. 2021-11-17 12:41:54,156 ERROR 8636 --- [inspectIT] [ Thread-0] r.i.o.core.config.InspectitEnvironment : instrumentation.rules.r_extract_pet_type.metrics => Metric 'pets/count' is not defined in inspectit.metrics.definitions! 2021-11-17 12:41:54,191 ERROR 8671 --- [inspectIT] [ Thread-0] r.i.o.core.config.InspectitEnvironment : Startup configuration is not valid! Using fallback configuration but listening for configuration updates...
Furthermore, the prometheus endpoint is dead, ie. not available. I'd expect that the this wrong config should be skipped and inspectIT would go on with the remaining instrumentation and keep surving the /metrics port. Even the default port 8888 is not served at all.
Hi @albersma69
yes, that's right. In the rule, you have to define the metric for which data should be exported and not its views. The views represents a "view" on a metric based on certain attributes, thus, the metric itself is responsible for collecting the data. Due to this, the rule must use the metric's name [pets]
instead of one of its views.
rules:
'r_extract_pet_type':
scopes:
...
metrics:
'[pets]':
...
Regarding the error: yes, that is the intended behavior of the agent. In case the configuration is invalid, it is rejected in order to prevent that data is recorded which my be invalid due to a broken or partial configuration. In this case, the last valid configuration will be used instead. In case this happens at startup, the agent falls back to a minimal configuration. That is also the reason that the Prometheus exporter is not started.
Too me this reads like works as designed. Additionally, there was no further feedback in several months. Hence, we close this issue.