mongodb_exporter icon indicating copy to clipboard operation
mongodb_exporter copied to clipboard

mongos is missing metrics, config nodes and shard nodes are OK

Open chj9 opened this issue 2 years ago • 5 comments

Describe the bug

Mongos missing metrics,the following is just one example. In fact, there are many metrics that do not have information, such as mongodb_instance_uptime_seconds

To Reproduce

My mongos yaml

net:
    bindIp: 0.0.0.0
    port: 27017

systemLog:
    logAppend: true

sharding:
    configDB: configRS/atai-testing-0:27019,atai-testing-1:27019,atai-testing-2:27019

My Pod yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: mongodb-exporter-router-0
  name: mongodb-exporter-router-0
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: mongodb-exporter-router-0
  template:
    metadata:
      labels:
        k8s-app: mongodb-exporter-router-0
    spec:
      containers:
        - args:
            - --log.level=info
            - --compatible-mode 
            - --mongodb.global-conn-pool
            - --mongodb.collstats-colls=turnright.*,turnrightForEncrypt.*,turnrightForVipQA2.*,turnrightForQA.*,turnrightForVip.*
            - --mongodb.indexstats-colls=turnright.*,turnrightForEncrypt.*,turnrightForVipQA2.*,turnrightForQA.*,turnrightForVip.*
            - --discovering-mode
            - --collector.dbstats
          env:
            - name: MONGODB_URI
              value: "mongodb://172.17.20.45:27017/"
          image: percona/mongodb_exporter:0.30.0
          imagePullPolicy: IfNotPresent
          name: mongodb-exporter
          ports:
            - containerPort: 9216
      restartPolicy: Always

---
kind: Service
apiVersion: v1
metadata:
  name: router-0
  labels:
    k8s-app: mongodb-exporter-router-0
spec:
  ports:
    - protocol: TCP
      port: 9216
      targetPort: 9216
  selector:
    k8s-app: mongodb-exporter-router-0
  type: ClusterIP

Request router metrics API curl http://router-0.mongodb-system:9216/metrics | grep mongodb_version_info Response

# HELP mongodb_version_info The server version
# TYPE mongodb_version_info gauge
mongodb_version_info{mongodb="server version is unavailable"} 1

version is server version is unavailable

Expected behavior

Request config or shards metrics API curl http://shard-1.mongodb-system:9216/metrics | grep mongodb_version_info Response

 # TYPE mongodb_version_info gauge
mongodb_version_info{mongodb="4.4.12"} 1

Logs

router logs

time="2022-03-02T03:21:31Z" level=warning msg="cannot create metrics for oplog: (IllegalOperation) Can't use 'local' database through mongos"
 time="2022-03-02T03:21:32Z" level=warning msg="cannot create metrics for oplog: (IllegalOperation) Can't use 'local' database through mongos"
 time="2022-03-02T03:21:41Z" level=warning msg="cannot create metrics for oplog: (IllegalOperation) Can't use 'local' database through mongos"

Environment

  • OS Centos 7.9
  • environment (docker, k8s, etc) K8S
  • MongoDB version 4.4.12
  • mongodb_exporter version 0.30.0 Additional context

docker exec -it mongodb-router mongo --host 172.17.20.45 --port 27017 Run command db.serverStatus() Response image

chj9 avatar Mar 02 '22 03:03 chj9

got the same problem with mongo version : 4.4.10-11 . and also got level=warning msg="cannot create metrics for oplog: (IllegalOperation) Can't use 'local' database through mongos" in the exporter log. Did you manage to make it work?

kriegmaster56 avatar Mar 15 '22 09:03 kriegmaster56

same problem whit mongod 4.2.7 . but there no any logs in the exporter, just print mongodb-exporter | time="2022-08-08T09:00:07Z" level=info msg="Starting HTTP server for http://:9216/metrics ..." source="server.go:140"

Codecaver avatar Aug 09 '22 02:08 Codecaver

got the same problem with mongo version : 4.4.15-15

jicki avatar Sep 15 '22 02:09 jicki

@chj9 have you you resolved the issue?

jicki avatar Sep 15 '22 02:09 jicki

@jicki you should add option --collect-all or specified one as https://github.com/percona/mongodb_exporter/blob/main/REFERENCE.md

hoangphuocbk avatar Dec 01 '22 13:12 hoangphuocbk