Clarify how kubernetes_sd_configs works for scraping pods, replicasets, etc.

[wstrange]

It was really non obvious to me how I would scrape a replica set where the targets are dynamic pods. i.e. how does Prom know how/where to contact the dynamic pods?

The blog that connected the dots for me was:

http://marselester.com/prometheus-on-kubernetes.html

 source_labels: [__meta_kubernetes_pod_label_app]
        regex: hello-server

Means "look for a pod that has meta data app=hello-server".

A short example snippet connecting these concepts together would really help.

[Typraeurion]

I still don’t understand how Prometheus discovers the right pods after reading the blog post above. E.g., how does it know to look at “hello:8000/metrics” and not, for example, “foo:8088/management/prometheus”; and how does it know to look for pods with the label “app: hello-server” and not, for example, “app: bar”?

[imranrazakhan]

@Typraeurion Hope below snippet helps you

scrape_configs:
  - job_name: 'activemq monitoring'
    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.
    # The HTTP resource path on which to fetch metrics from targets.
    metrics_path: '/activemq-monitoring-app/metrics'