Provide/Document a ServiceMonitor

[ba-work]

Image I'm using: Not relevant.

Issue or Feature Request: I would imagine most people will have a preexisting prometheus running and thus yamlgen/telemetry/promethues-resources.yaml is not really that helpful. This is what I use, maybe there is something better?

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: bottlerocker-update-operator
  namespace: brupop-bottlerocket-aws
spec:
  jobLabel: jobLabel
  namespaceSelector:
    matchNames:
      - brupop-bottlerocket-aws
  selector:
    matchLabels:
      app.kubernetes.io/part-of: brupop
  endpoints:
    - path: /metrics

[gthao313]

@ba-work Thanks for opening this issue, Bottlerocket is investigating on it and will reach back to you soon.

[somnusfish]

@ba-work Thanks for opening this issue. yamlgen/telemetry/promethues-resources.yaml is just a sample for using the prometheus metrics, the user should feel free to use whatever way they like with the endpoint.

The sample that you provided definitely looks more simple. I think our next step could be creating a telemetry/examples/ folder and put different sample config under different use case in that folder.

Before moving forward putting your sample, would you mind provide more details to us so we can better document your sample?

1. How did you install Prometheus, is there any special config?
2. Would this setting persist data after rebooting?

Thanks

[ba-work]

@somnusfish Prometheus is installed using prometheus operator from the kube-prometheus-stack chart here: https://github.com/prometheus-community/helm-charts/blob/main/charts/kube-prometheus-stack/README.md

It will even persist data (and keep data in the same time series) if you fully delete and resinstall brupop since the selector's labels will continue to match. The example I posted does not impose any retention options so it will inherit whatever is set globally Prometheus.

[jpmcb]

I would imagine most people will have a preexisting prometheus running and thus yamlgen/telemetry/promethues-resources.yaml is not really that helpful.

True - our yamlgen/telemetry/prometheus-resources.yaml exists simply as an example for users who might want to deploy prometheus to their cluster and get observbility metrics on brupop.

We did add:

annotations:
  prometheus.io/port: '8080'
  prometheus.io/scrape: 'true'

to our deployments so prometheus deployed on a cluster with the default/basic scrape config can automatically start picking up metrics.

Closing this issue since providing a sensible prometheus / metrics store solution is out of scope for the bottlerocket-update-operator.