ServiceMonitor (for prometheus operator)

[leogr]

In order to make kube-notary discovered by the prometheus operator:

• the endpoint resource needs the following label:

  labels:
    k8s-app: kube-notary

• the ServiceMonitor resource has to be created, as following:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: kube-notary
spec:
  endpoints:
  - interval: 15s
    port: metrics-port
  selector:
    matchLabels:
      k8s-app: kube-notary

TODOs:

• the required label could be set in the default setup
• this configuration should be added to the repo

[dzlabsch]

I solved that differently by adding an additional Scrape Config to Prometheus operator.

create config file

- job_name: 'kubernetes-pods'
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
action: replace
target_label: __metrics_path__
regex: (.+)
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
action: replace
regex: ([^:]+)(?::\d+)?;(\d+)
replacement: $1:$2
target_label: __address__
- action: labelmap
regex: __meta_kubernetes_pod_label_(.+)
- source_labels: [__meta_kubernetes_namespace]
action: replace
target_label: kubernetes_namespace
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name

create a secret

kubectl create secret generic additional-scrape-configs --from-file=prometheus-additional.yaml --dry-run -oyaml > additional-scrape-configs.yaml

apply the config to the namespace prometheus operator is running

kubectl apply -f additional-scrape-configs.yaml --namespace=monitoring

edit the runtime

kubectl edit Prometheus kube-prometheus --namespace=monitoring

under spec

spec:
additionalScrapeConfigs:
key: prometheus-additional.yaml
name: additional-scrape-configs