Adding timestamps and total duration

[lgluisgaspar]

Adding timestamps and total duration into bin/ghe-backup and timestamps share/github-backup-utils/bm.sh

Example of benchmark file log

2022-09-19 15:30:33 - ghe-backup-store-version took 0s
2022-09-19 15:30:35 - ghe-backup-settings took 2s
2022-09-19 15:30:35 - ghe-export-authorized-keys took 0s
2022-09-19 15:30:35 - ghe-export-ssh-host-keys took 0s
2022-09-19 15:30:46 - ghe-backup-mysql-binary took 11s
2022-09-19 15:30:46 - ghe-backup-mysql took 11s
2022-09-19 15:30:47 - ghe-backup-actions took 0s
2022-09-19 15:30:47 - ghe-backup-minio took 0s
2022-09-19 15:30:49 - ghe-backup-redis took 2s
2022-09-19 15:30:49 - ghe-backup-es-audit-log took 0s
2022-09-19 15:30:51 - ghe-backup-repositories - Generating routes took 2s
2022-09-19 15:30:51 - ghe-backup-repositories - Fetching routes took 0s
2022-09-19 15:30:51 - ghe-backup-repositories - Processing routes took 0s
2022-09-19 15:30:53 - ghe-backup-repositories - Repo sync took 2s
2022-09-19 15:30:53 - ghe-backup-repositories - Special Data Directories Sync took 0s
2022-09-19 15:30:53 - ghe-backup-repositories - Verifying Routes took 0s
2022-09-19 15:30:53 - ghe-backup-repositories took 4s
2022-09-19 15:30:54 - ghe-backup-pages - lgluisgaspar.sickbaylab.com took 1s
2022-09-19 15:30:54 - ghe-backup-pages took 1s
2022-09-19 15:30:56 - ghe-backup-storage - Generating routes took 2s
2022-09-19 15:30:56 - ghe-backup-storage - Fetching routes took 0s
2022-09-19 15:30:56 - ghe-backup-storage - Processing routes took 0s
2022-09-19 15:30:56 - ghe-backup-git-hooks took 0s
2022-09-19 15:30:57 - ghe-backup-es-rsync took 1s

Example of backup file log |grep INFO

2022-09-19 15:43:46 - INFO - Starting backup of 🕵🏻‍♂️ with backup-utils v3.6.0 in snapshot 20220919T154346
2022-09-19 15:43:47 - INFO - Backing up GitHub settings ...
2022-09-19 15:43:49 - INFO - Backing up SSH authorized keys ...
2022-09-19 15:43:49 - INFO - Backing up SSH host keys ...
2022-09-19 15:43:49 - INFO - Backing up MySQL databases ...
2022-09-19 15:44:00 - INFO - Backing up MSSQL databases ...
2022-09-19 15:44:42 - INFO - Backing up Actions data ...
2022-09-19 15:44:42 - INFO - Backing up Minio data ...
2022-09-19 15:44:43 - INFO - Backing up Redis database ...
2022-09-19 15:44:44 - INFO - Backing up audit log ...
2022-09-19 15:44:44 - INFO - Backing up Git repositories ...
2022-09-19 15:44:48 - INFO - Backing up GitHub Pages artifacts ...
2022-09-19 15:44:49 - INFO - Backing up storage data ...
2022-09-19 15:44:51 - INFO - Backing up custom Git hooks ...
2022-09-19 15:44:51 - INFO - Backing up Elasticsearch indices ...
2022-09-19 15:44:52 - INFO - Completed backup of 🕵🏻‍♂️ in snapshot 20220919T154346 at 15:44:52
2022-09-19 15:44:52 - INFO - Total duration time: 66 seconds
2022-09-19 15:44:52 - INFO - Checking for leaked ssh keys ...

[lgluisgaspar]

The date format can be changed on a better solution.

[donal]

@lgluisgaspar Thanks for submitting this — noting that you've incorporated the change in https://github.com/github/backup-utils/pull/956.

[lgluisgaspar]

After i edited ghe-restore, i noticed the variables $END_TIME and $START_TIME, i renamed same in ghe-backup. I did not notice previously.

[0983809952]

Adding timestamps and total duration into bin/ghe-backup and timestamps share/github-backup-utils/bm.sh

Example of benchmark file log

2022-09-19 15:30:33 - ghe-backup-store-version took 0s
2022-09-19 15:30:35 - ghe-backup-settings took 2s
2022-09-19 15:30:35 - ghe-export-authorized-keys took 0s
2022-09-19 15:30:35 - ghe-export-ssh-host-keys took 0s
2022-09-19 15:30:46 - ghe-backup-mysql-binary took 11s
2022-09-19 15:30:46 - ghe-backup-mysql took 11s
2022-09-19 15:30:47 - ghe-backup-actions took 0s
2022-09-19 15:30:47 - ghe-backup-minio took 0s
2022-09-19 15:30:49 - ghe-backup-redis took 2s
2022-09-19 15:30:49 - ghe-backup-es-audit-log took 0s
2022-09-19 15:30:51 - ghe-backup-repositories - Generating routes took 2s
2022-09-19 15:30:51 - ghe-backup-repositories - Fetching routes took 0s
2022-09-19 15:30:51 - ghe-backup-repositories - Processing routes took 0s
2022-09-19 15:30:53 - ghe-backup-repositories - Repo sync took 2s
2022-09-19 15:30:53 - ghe-backup-repositories - Special Data Directories Sync took 0s
2022-09-19 15:30:53 - ghe-backup-repositories - Verifying Routes took 0s
2022-09-19 15:30:53 - ghe-backup-repositories took 4s
2022-09-19 15:30:54 - ghe-backup-pages - lgluisgaspar.sickbaylab.com took 1s
2022-09-19 15:30:54 - ghe-backup-pages took 1s
2022-09-19 15:30:56 - ghe-backup-storage - Generating routes took 2s
2022-09-19 15:30:56 - ghe-backup-storage - Fetching routes took 0s
2022-09-19 15:30:56 - ghe-backup-storage - Processing routes took 0s
2022-09-19 15:30:56 - ghe-backup-git-hooks took 0s
2022-09-19 15:30:57 - ghe-backup-es-rsync took 1s

Example of backup file log |grep INFO

2022-09-19 15:43:46 - INFO - Starting backup of 🕵🏻‍♂️ with backup-utils v3.6.0 in snapshot 20220919T154346
2022-09-19 15:43:47 - INFO - Backing up GitHub settings ...
2022-09-19 15:43:49 - INFO - Backing up SSH authorized keys ...
2022-09-19 15:43:49 - INFO - Backing up SSH host keys ...
2022-09-19 15:43:49 - INFO - Backing up MySQL databases ...
2022-09-19 15:44:00 - INFO - Backing up MSSQL databases ...
2022-09-19 15:44:42 - INFO - Backing up Actions data ...
2022-09-19 15:44:42 - INFO - Backing up Minio data ...
2022-09-19 15:44:43 - INFO - Backing up Redis database ...
2022-09-19 15:44:44 - INFO - Backing up audit log ...
2022-09-19 15:44:44 - INFO - Backing up Git repositories ...
2022-09-19 15:44:48 - INFO - Backing up GitHub Pages artifacts ...
2022-09-19 15:44:49 - INFO - Backing up storage data ...
2022-09-19 15:44:51 - INFO - Backing up custom Git hooks ...
2022-09-19 15:44:51 - INFO - Backing up Elasticsearch indices ...
2022-09-19 15:44:52 - INFO - Completed backup of 🕵🏻‍♂️ in snapshot 20220919T154346 at 15:44:52
2022-09-19 15:44:52 - INFO - Total duration time: 66 seconds
2022-09-19 15:44:52 - INFO - Checking for leaked ssh keys ...

[djdefi]

Looks like a duplicate of https://github.com/github/backup-utils/pull/711

cc @gamefiend to consider which way to go here as part of logging improvements

[gamefiend]

Looks like a duplicate of #711

cc @gamefiend to consider which way to go here as part of logging improvements

Thanks for pinging me on this @djdefi.

This looks really good! One thing I'd like to ask: can we pull this PR into the private repo to better coordinate efforts?

[gamefiend]

logging was added as part of https://github.com/github/ghes/issues/5440 and will be introduced with our next release. Closing this PR.