No Backups Found: WAL-G uses `lastModified` attribute instead of timestamp from json with a metadata

[dkrylovdotcom]

Database name

Postgres 14

Issue description

Our stack

• We have kubernetes clusters, where the users run their Postgres clusters (using zalando postgres operator).
• To provide DB backup ability, we are using WAL-G with S3 as a storage.
• We have Corporate Backup System (CBS - CommVault) that is the end-storage (arching data on a magnetic tape) for the user data.
• In our case s3 is not a reliable storage, so the recovery from CBS could be performed in someday

The problem

• All new files in S3 have the current lastModified timestamp.
• WAL-G uses lastModified as a timestamp to restore.
• If we perform a restore from CBS, WAL-G could find any backups due to a new lastModified ;(
• At the same time, WAL-G creates json files with a metadata, which have backup timestamp: s3://postgres-backup/spilo/pg-cluster/wal/14/basebackups_005/base_0000007*******_backup_stop_sentinel.json (I assume that this timestamp from json with a metadata should be used, not lastModified)

Steps to reproduce

The Backup flow is the next:

1. Users runs their Postgres cluster and configure WAL-G to send backups to S3
2. WAL-G sending backups to S3 Bucket
3. Let's imagine that is no data in S3 at someday, than the user need to request restore to S3 from CBS
4. Everything is restored, but here is a problem: files in S3 created with lastModified timestamp related to the current date
5. We are facing with No backups found error

Message link to a community chat: https://t.me/c/1404070833/3882