percona-backup-mongodb icon indicating copy to clipboard operation
percona-backup-mongodb copied to clipboard

Published 20 hours ago • verified publisher icon percona

Physical backups improvements

Open dAdAbird opened this issue 3 years ago • 12 comments

Added new command pbm describe-restore

Outputs detailed information on the given restore.

$ pbm describe-restore 2022-08-03T09:25:54.4242858Z
name: "2022-08-03T09:25:54.4242858Z"
backup: "2022-07-28T12:10:13Z"
type: logical
status: done
replsets:
- name: rs1
  status: done
  last_transition_time: "2022-08-03T09:26:02Z"
- name: cfg
  status: done
  last_transition_time: "2022-08-03T09:26:03Z"
- name: rs2
  status: done
  last_transition_time: "2022-08-03T09:26:03Z"
opid: 62ea3f2202ad53f3e9e8ea59
start: "2022-08-03T09:25:54Z"
last_transition_time: "2022-08-03T09:26:04Z"

It also has a flag -c (--config) which works only with physical restores. It allows pointing a pbm config so PBM can read restore status on storage (mongod during physical restore is down). Example: pbm describe-restore -c /etc/pbm/conf.yaml 2022-08-02T18:50:35.1889332Z

Use restore name instead of backup's name

In a number of places (pbm list --restore, pbm logs) restore was identified bu the backup's name. Which is inconvenient as we may have multiple restores from the same backup. Now rather the restore name is used everywhere. pbm restore also is displaying the restore name.

Consider WiredTiger options during the restore (PBM-909)

During physical backup, store a WT config in the backup's meta. The restore will apply that config during internal temporary mongod starts.

Physical restores

  • Improved tmp port peeking.
  • Agents send heartbeats via storage. Hence pbm is aware of system longevity during restore. (PBM-813)
  • Improved agents' sync and error reporting.
  • Internal tmp mongod starts now write logs to pbm.restore.log inside dbpath. This file is removed after the restore only if it's finished successfully. (PBM-898, PBM-812)

Fixes https://jira.percona.com/browse/PBM-813 Fixes https://jira.percona.com/browse/PBM-898, https://jira.percona.com/browse/PBM-812 Fixes https://jira.percona.com/browse/PBM-909

dAdAbird avatar Jul 26 '22 15:07 dAdAbird

Test name Logical Physical
New cluster psmdb 4.4 passed -
New cluster psmdb 4.2 passed -
Sharded psmdb 4.2 passed failed
Sharded psmdb 4.4 passed passed
Sharded psmdb 5.0 failed passed
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 4.4 passed passed
Single-node psmdb 4.4 failed passed
Single-node psmdb 4.2 failed passed
Single-node psmdb 5.0 passed passed
Non-sharded psmdb 5.0 passed passed

JNKPercona avatar Jul 26 '22 19:07 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed passed
Sharded psmdb 4.4 failed passed
Sharded psmdb 5.0 failed passed
New cluster psmdb 4.2 passed -
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 failed -
Non-sharded psmdb 4.2 passed passed
Non-sharded psmdb 4.4 passed passed
Non-sharded psmdb 5.0 passed passed
Single-node psmdb 4.2 passed passed
Single-node psmdb 4.4 failed passed
Single-node psmdb 5.0 passed passed

JNKPercona avatar Jul 27 '22 12:07 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 failed -
Sharded psmdb 5.0 failed passed
Sharded psmdb 4.4 failed passed
Sharded psmdb 4.2 failed failed

JNKPercona avatar Jul 27 '22 13:07 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed passed
Sharded psmdb 4.4 failed passed
Sharded psmdb 5.0 failed failed
New cluster psmdb 4.2 failed -
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 5.0 failed passed
Non-sharded psmdb 4.4 failed failed
Single-node psmdb 5.0 failed passed
Single-node psmdb 4.2 failed passed
Single-node psmdb 4.4 failed passed

JNKPercona avatar Jul 27 '22 17:07 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed failed
Sharded psmdb 4.4 failed failed
New cluster psmdb 4.2 passed -
Sharded psmdb 5.0 failed failed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 failed -
Non-sharded psmdb 4.2 failed failed
Non-sharded psmdb 4.4 failed failed
Non-sharded psmdb 5.0 failed failed
Single-node psmdb 4.2 failed failed
Single-node psmdb 4.4 failed failed
Single-node psmdb 5.0 failed failed

JNKPercona avatar Jul 28 '22 18:07 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 failed -
Sharded psmdb 4.2 failed failed
Sharded psmdb 4.4 failed -
Sharded psmdb 5.0 failed -
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 4.4 failed passed
Non-sharded psmdb 5.0 failed passed
Single-node psmdb 4.2 failed passed
Single-node psmdb 4.4 failed passed
Single-node psmdb 5.0 failed passed

JNKPercona avatar Aug 02 '22 19:08 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed passed
New cluster psmdb 4.4 passed -
New cluster psmdb 4.2 failed -
Sharded psmdb 4.4 failed passed
New cluster psmdb 5.0 passed -
Sharded psmdb 5.0 failed passed
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 4.4 failed passed
Single-node psmdb 4.2 failed failed
Single-node psmdb 4.4 failed passed
Non-sharded psmdb 5.0 failed passed
Single-node psmdb 5.0 failed passed

JNKPercona avatar Aug 03 '22 09:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 passed -
Sharded psmdb 4.2 failed passed
Sharded psmdb 4.4 failed passed
Sharded psmdb 5.0 failed passed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 failed -
Non-sharded psmdb 4.2 failed failed
Non-sharded psmdb 4.4 failed passed
Non-sharded psmdb 5.0 failed failed
Single-node psmdb 4.2 failed passed
Single-node psmdb 4.4 failed passed
Single-node psmdb 5.0 failed passed

JNKPercona avatar Aug 03 '22 10:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 failed -
New cluster psmdb 5.0 failed -
Non-sharded psmdb 4.2 failed failed
New cluster psmdb 4.4 passed -
Non-sharded psmdb 4.4 failed passed
Single-node psmdb 4.4 failed passed
Non-sharded psmdb 5.0 failed failed
Single-node psmdb 4.2 failed failed
Single-node psmdb 5.0 failed failed

JNKPercona avatar Aug 03 '22 10:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 passed -
Sharded psmdb 4.2 failed passed
Sharded psmdb 4.4 failed passed
Sharded psmdb 5.0 failed passed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 4.4 failed passed
Single-node psmdb 4.4 failed passed
Single-node psmdb 4.2 failed passed
Non-sharded psmdb 5.0 failed passed
Single-node psmdb 5.0 failed passed

JNKPercona avatar Aug 03 '22 12:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 passed -
Sharded psmdb 4.2 failed passed
Sharded psmdb 4.4 failed passed
Sharded psmdb 5.0 failed passed
New cluster psmdb 4.4 passed -
Non-sharded psmdb 4.2 failed passed
New cluster psmdb 5.0 passed -
Non-sharded psmdb 5.0 failed passed
Non-sharded psmdb 4.4 failed passed
Single-node psmdb 4.2 failed passed
Single-node psmdb 4.4 failed passed
Single-node psmdb 5.0 failed failed

JNKPercona avatar Aug 04 '22 15:08 JNKPercona

Test name Logical Physical
Sharded psmdb 4.4 passed passed
Sharded psmdb 5.0 failed passed
Sharded psmdb 4.2 passed passed

JNKPercona avatar Aug 04 '22 18:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 failed -
Sharded psmdb 4.2 failed failed
Sharded psmdb 4.4 failed failed
Sharded psmdb 5.0 failed failed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 4.4 failed passed
Non-sharded psmdb 5.0 failed passed
Single-node psmdb 4.2 failed passed
Single-node psmdb 5.0 failed passed
Single-node psmdb 4.4 failed passed

JNKPercona avatar Aug 20 '22 03:08 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed passed
New cluster psmdb 4.2 failed -
New cluster psmdb 4.4 passed -
Sharded psmdb 4.4 passed passed
Sharded psmdb 5.0 passed failed
New cluster psmdb 5.0 failed -
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 4.4 failed failed
Non-sharded psmdb 5.0 passed failed
Single-node psmdb 4.2 passed passed
Single-node psmdb 5.0 passed passed
Single-node psmdb 4.4 passed passed

JNKPercona avatar Aug 21 '22 13:08 JNKPercona

Test name Logical Physical
Non-sharded psmdb 4.2 passed failed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 passed -
New cluster psmdb 4.2 failed -
Sharded psmdb 4.4 passed failed
Sharded psmdb 4.2 failed failed
Sharded psmdb 5.0 failed failed
Non-sharded psmdb 4.4 failed failed
Non-sharded psmdb 5.0 failed failed
Single-node psmdb 4.2 failed failed
Single-node psmdb 5.0 passed failed
Single-node psmdb 4.4 passed failed

JNKPercona avatar Aug 22 '22 18:08 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed passed
Sharded psmdb 4.4 failed failed
Sharded psmdb 5.0 failed passed
New cluster psmdb 4.2 passed -
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 failed -
Non-sharded psmdb 4.4 passed failed
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 5.0 passed failed
Single-node psmdb 4.2 failed failed
Single-node psmdb 4.4 passed failed
Single-node psmdb 5.0 passed failed

JNKPercona avatar Aug 23 '22 18:08 JNKPercona

Test name Logical Physical
Sharded psmdb 4.2 failed failed
New cluster psmdb 4.2 passed -
New cluster psmdb 5.0 failed -
New cluster psmdb 4.4 passed -
Non-sharded psmdb 4.2 passed passed
Non-sharded psmdb 4.4 passed passed
Non-sharded psmdb 5.0 passed failed
Single-node psmdb 4.2 passed passed
Single-node psmdb 4.4 passed passed
Single-node psmdb 5.0 passed failed
Sharded psmdb 4.4 failed failed
Sharded psmdb 5.0 failed failed

JNKPercona avatar Aug 24 '22 16:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 passed -
Sharded psmdb 4.2 failed failed
New cluster psmdb 4.4 passed -
New cluster psmdb 5.0 failed -
Non-sharded psmdb 4.2 passed passed
Single-node psmdb 4.2 passed failed
Non-sharded psmdb 4.4 passed passed
Non-sharded psmdb 5.0 failed passed
Single-node psmdb 4.4 passed failed
Single-node psmdb 5.0 passed failed
Sharded psmdb 4.4 failed failed
Sharded psmdb 5.0 failed failed

JNKPercona avatar Aug 24 '22 17:08 JNKPercona

Test name Logical Physical
New cluster psmdb 4.2 passed -
New cluster psmdb 4.4 passed -
Sharded psmdb 4.2 failed failed
Sharded psmdb 4.4 failed failed
New cluster psmdb 5.0 passed -
Non-sharded psmdb 4.2 failed passed
Non-sharded psmdb 4.4 passed failed
Non-sharded psmdb 5.0 failed passed
Single-node psmdb 4.2 passed passed
Single-node psmdb 4.4 passed passed
Single-node psmdb 5.0 failed failed
Sharded psmdb 5.0 failed failed

JNKPercona avatar Aug 25 '22 13:08 JNKPercona