percona-backup-mongodb icon indicating copy to clipboard operation
percona-backup-mongodb copied to clipboard

PBM-902: fix sharded cluster detection on mongo 6.0

Open dAdAbird opened this issue 3 years ago • 3 comments

Check for db.adminCommand({ getCmdLineOpts: 1}).parsed.sharding.clusterRole if ConfigServerState is anavaliable.

Fixes https://jira.percona.com/browse/PBM-902

dAdAbird avatar Aug 03 '22 14:08 dAdAbird

Test name Logical Physical
New cluster psmdb 4.2 passed -
Sharded psmdb 4.2 passed passed
Sharded psmdb 4.4 passed passed
Sharded psmdb 5.0 failed passed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 passed passed
Non-sharded psmdb 4.4 failed passed
Non-sharded psmdb 5.0 failed passed
Single-node psmdb 4.4 failed passed
Single-node psmdb 4.2 passed passed
Single-node psmdb 5.0 failed passed

JNKPercona avatar Aug 03 '22 17:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 passed -
Sharded psmdb 4.2 passed passed
Sharded psmdb 4.4 passed passed
Sharded psmdb 5.0 failed passed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 failed passed
Single-node psmdb 4.2 passed passed
Non-sharded psmdb 5.0 passed passed
Non-sharded psmdb 4.4 passed passed
Single-node psmdb 4.4 passed passed
Single-node psmdb 5.0 passed passed

JNKPercona avatar Aug 08 '22 06:08 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed passed
Sharded psmdb 5.0 failed passed
Sharded psmdb 4.4 failed passed
New cluster psmdb 4.2 passed -
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.4 passed passed
Non-sharded psmdb 4.2 failed passed
Single-node psmdb 4.4 passed passed
Single-node psmdb 5.0 passed passed
Single-node psmdb 4.2 passed passed
Non-sharded psmdb 5.0 passed passed

JNKPercona avatar Aug 08 '22 22:08 JNKPercona

We are waiting for this change a lot, as it seems that without it our 6.0 cluster with config servers colocated with shards on same instance shows the error status for all shards without logging anything helpful in the agent logs:

pbm --mongodb-uri "mongodb://pbm:<pwd>@localhost:27018/?authSource=admin" status
Cluster:
========
nonprod-config:
  - nonprod-config/nonprod-mongodb1.nonprod.example.com:27018: pbm-agent v1.8.1 OK
  - nonprod-config/nonprod-mongodb2.nonprod.example.com:27018: pbm-agent v1.8.1 OK
  - nonprod-config/nonprod-mongodb3.nonprod.example.com:27018: pbm-agent v1.8.1 OK
nonprod-shard-1:
  - nonprod-shard-1/nonprod-mongodb1.nonprod.example.com:27020: pbm-agent NOT FOUND FAILED status:
  - nonprod-shard-1/nonprod-mongodb2.nonprod.example.com:27020: pbm-agent NOT FOUND FAILED status:
  - nonprod-shard-1/nonprod-mongodb3.nonprod.example.com:27020: pbm-agent NOT FOUND FAILED status:
nonprod-shard-2:
  - nonprod-shard-2/nonprod-mongodb1.nonprod.example.com:27021: pbm-agent NOT FOUND FAILED status:
  - nonprod-shard-2/nonprod-mongodb2.nonprod.example.com:27021: pbm-agent NOT FOUND FAILED status:
  - nonprod-shard-2/nonprod-mongodb3.nonprod.example.com:27021: pbm-agent NOT FOUND FAILED status:
nonprod-shard-3:
  - nonprod-shard-3/nonprod-mongodb1.nonprod.example.com:27022: pbm-agent NOT FOUND FAILED status:
  - nonprod-shard-3/nonprod-mongodb2.nonprod.example.com:27022: pbm-agent NOT FOUND FAILED status:
  - nonprod-shard-3/nonprod-mongodb3.nonprod.example.com:27022: pbm-agent NOT FOUND FAILED status:

daniilyar-fnddev avatar Aug 17 '22 00:08 daniilyar-fnddev

I confirm this fixes the problem. I was also unable to have pbm-agent running correctly on a 6.0 sharded cluster:

configReplicaSet:
  - configReplicaSet/mongo-cfg01-sva:27019: pbm-agent v1.8.1 OK
  - configReplicaSet/mongo-cfg02-sva:27019: pbm-agent v1.8.1 OK
  - configReplicaSet/mongo-cfg03-sva:27019: pbm-agent v1.8.1 OK
replicaSet1:
  - replicaSet1/mongo01-sva:27018: pbm-agent NOT FOUND FAILED status:
  - replicaSet1/mongo02-sva:27018: pbm-agent NOT FOUND FAILED status:
(..)

After compiling pbm-agent with this fix, it worked:

configReplicaSet:
  - configReplicaSet/mongo-cfg01-sva:27019: pbm-agent v1.8.1 OK
  - configReplicaSet/mongo-cfg02-sva:27019: pbm-agent v1.8.1 OK
  - configReplicaSet/mongo-cfg03-sva:27019: pbm-agent v1.8.1 OK
replicaSet1:
  - replicaSet1/mongo01-sva:27018: pbm-agent v1.8.1 OK
  - replicaSet1/mongo02-sva:27018: pbm-agent v1.8.1 OK
(..)

Any idead when can we expect a 1.8.2 release with this fix?

bragatto avatar Aug 23 '22 11:08 bragatto