Prometheus & grafana integration in helm chart

[maxpain]

Proposal

Provide a seamless way to monitor managed MongoDB databases using PodMonitor for scraping, PrometheusRule for alerting, and ConfigMap with the grafana dashboard.

The helm chart should automatically provide it. Many operators, such as CNPG, integrate monitoring in their Helm charts.

Use-Case

No response

Is this a feature you are interested in implementing yourself?

No

Anything else?

No response

[radoslawszulgo]

Hi @maxpain Can you please provide some more details on your choice of tools? Have you already seen our PMM that may solve these use cases of monitoring, alerting, and dashboarding? Details: https://docs.percona.com/percona-operator-for-mongodb/monitoring-tutorial.html?h=pmm#install-pmm-client

[bitfisher]

Just came across this because we'd need monitoring using Prometheus with PodMonitor too.

I believe that's a very common use-case for monitoring/alerting in k8s.

It's not clear to me if pmm-client can help here.

Is it possible to set pmm.enabled to true without a pmm-server running and let prometheus scrape metrics?

[bitfisher]

At the moment i'm using sidecar and podmonitor to achieve prom scraping.

...
      sidecars:
        - name: metrics
          image: percona/mongodb_exporter:0.47.1
          securityContext:
            runAsNonRoot: true
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
          resources:
            requests:
              cpu: 10m
              memory: 32Mi
            limits:
              cpu: 250m
              memory: 128Mi
          ports:
            - name: metrics
              containerPort: 9216
              protocol: TCP
          env:
            - name: MONGODB_USER
              valueFrom:
                secretKeyRef:
                  name: sample-psmdb-users
                  key: MONGODB_CLUSTER_MONITOR_USER
            - name: MONGODB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: sample-psmdb-users
                  key: MONGODB_CLUSTER_MONITOR_PASSWORD
          args:
            - "--discovering-mode"
            - "--collect-all"
            - "--log.level=info"
            - "--mongodb.uri=mongodb://localhost:27017"
...
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: sample-psmdb
  namespace: mas-test
  labels:
    app.kubernetes.io/instance: "sample-psmdb"
    app.kubernetes.io/component: "mongod"
    app.kubernetes.io/part-of: "percona-server-mongodb"
spec:
  selector:
    matchLabels:
      app.kubernetes.io/instance: "sample-psmdb"
      app.kubernetes.io/component: "mongod"
  podMetricsEndpoints:
    - port: metrics
      interval: 30s
      scrapeTimeout: 10s
  namespaceSelector:
    matchNames:
      - mas-test

Please let me know, if we can achieve something similar using pmm-client.