fdbrestore fails with: Missing backup data

[brainpicture]

Trying to restore backup which was made on 6.0.18 My version is 6.1.8

The error is:

Tag: default  UID: a713aaec9e048425744647084b843be5  State: starting  Blocks: 0/0  BlocksInProgress: 0  Files: 0  BytesWritten
: 0  ApplyVersionLag: 0  LastError: ''Missing backup data' on 'restore_start'' 12s ago.

I am performing:

fdbrestore start -r file:///home/$USER/fdb.backup/backup-2019-07-06-22-08-23.328692 --dest_cluster_file ./fdb.cluster -w

what am I doing wrong, or should I have the same version of FoundationDB with which the backup was performed?

After updating source db I still have same error, even when both versions are 6.1.8. The only difference is the replication factor, how am I supposed to move data from one db to another if replication factors a different?

When I trying to import data via disaster recovery I got:

fdbdr status
SIGNAL: Segmentation fault (11)
Trace: addr2line -e fdbdr.debug -p -C -f -i 0x9d2de5 0x7faf1b8d60e0 0x5a81cf 0x5ac04d 0x478fd0 0x535b7b 0x82df48 0x6209c0 0x75fe33 0x76017b 0x74b248 0x74b54d 0x7e52f0 0x752129 0x7e52f0 0x791ec7 0x792010 0x76bfd0 0x777445 0x77bdd8 0x77bfdd 0x797dc0 0x71f018 0x927495 0x927819 0x478f40 0xa19e8a 0x74046e 0x423c15 0x7faf1b03a2e1

command:

fdbdr start -d ./fdb.cluster  -s /etc/foundationdb/fdb.cluster

[satherton]

Backups written with 6.0.18 are compatible with all newer versions to date, so you're good there.

The fdbrestore start command verifies that the backup contains the required files to restore to your selected version - it verifies presence but not contents - and then queues the restore job into the database for the backup_agent processes to work on. So note that it is the the backup_agent processes that do the actual reading of backup files, so they must have access to the files using the same path you provided to fdbrestore start.

In your case, it appears that fdbrestore start completed successfully and queued the job, but then the backup status you are getting from the cluster says data is missing, so my first guess is that the path you provided can't be accessed (due to it not being mounted, or possibly also a permission problem) from the host on which backup_agent is running.

Replication factor (as well as almost all other configuration options such as storage engine, log count, proxy count, etc) does not matter between clusters in a DR pair or between clusters generating vs reading a file-based backup.

To be clear, fdbdr and fdbrestore are completely unrelated. That said, fdbdr status should not crash of course, so can you create another issue for that?

[breign]

foundationdb-clients 6.1.12-1 amd64 FoundationDB clients and library foundationdb-server 6.1.12-1 amd64 FoundationDB server

Same problem. No document layer and the database is empty.

fdbbackup describe -d file:///tmp/fdbackup/backup-2019-09-06-15-53-19.858736

URL: file:///tmp/fdbackup/backup-2019-09-06-15-53-19.858736 Restorable: true Snapshot: startVersion=3651522374563 (maxLogEnd -0.00 days) endVersion=3651543294471 (maxLogEnd -0.00 days) totalBytes=1037721404 restorable=true expiredPct=0.00 SnapshotBytes: 1037721404 MinLogBeginVersion: 3651522159099 (maxLogEnd -0.00 days) ContiguousLogEndVersion: 3651562159099 (maxLogEnd -0.00 days) MaxLogEndVersion: 3651562159099 (maxLogEnd -0.00 days) MinRestorableVersion: 3651543294471 (maxLogEnd -0.00 days) MaxRestorableVersion: 3651562159098 (maxLogEnd -0.00 days)

fdbrestore start -w --dest_cluster_file /etc/foundationdb/fdb.cluster -r /tmp/fdbackup/backup-2019-09-06-15-53-19.858736

No restore target version given, will use maximum restorable version from backup description. Using target restore version 3651562159098 Backup Description URL: file:///tmp/fdbackup/backup-2019-09-06-15-53-19.858736 Restorable: true Snapshot: startVersion=3651522374563 (maxLogEnd -0.00 days) endVersion=3651543294471 (maxLogEnd -0.00 days) totalBytes=1037721404 restorable=true expiredPct=0.00 SnapshotBytes: 1037721404 MinLogBeginVersion: 3651522159099 (maxLogEnd -0.00 days) ContiguousLogEndVersion: 3651562159099 (maxLogEnd -0.00 days) MaxLogEndVersion: 3651562159099 (maxLogEnd -0.00 days) MinRestorableVersion: 3651543294471 (maxLogEnd -0.00 days) MaxRestorableVersion: 3651562159098 (maxLogEnd -0.00 days) ERROR: Missing backup data Fatal Error: Missing backup data

[satherton]

Please try running fdbrestore start --dryrun --log --dest_cluster_file /etc/foundationdb/fdb.cluster -r /tmp/fdbackup/backup-2019-09-06-15-53-19.858736

and look at the trace log file produced for error messages.

[xiongbl526]

Backups written with 6.0.18 are compatible with all newer versions to date, so you're good there.

The fdbrestore start command verifies that the backup contains the required files to restore to your selected version - it verifies presence but not contents - and then queues the restore job into the database for the backup_agent processes to work on. So note that it is the the backup_agent processes that do the actual reading of backup files, so they must have access to the files using the same path you provided to fdbrestore start.

In your case, it appears that fdbrestore start completed successfully and queued the job, but then the backup status you are getting from the cluster says data is missing, so my first guess is that the path you provided can't be accessed (due to it not being mounted, or possibly also a permission problem) from the host on which backup_agent is running.

Replication factor (as well as almost all other configuration options such as storage engine, log count, proxy count, etc) does not matter between clusters in a DR pair or between clusters generating vs reading a file-based backup.

To be clear, fdbdr and fdbrestore are completely unrelated. That said, fdbdr status should not crash of course, so can you create another issue for that?

Hi satherton, I also encountered the same problem. I am using FoundationDB version 7.1.15. I have already executed "chmod 777 /tmp/backup-2024-09-26-09-35-22.992383/" command. When I execute "fdbrestore start -r 'file:///tmp/backup-2024-09-26-09-35-22.992383' --dest-cluster-file=/home/newclusterfile/file --log" command, I get the following error message when executing the "status" command. Tag: default UID: 0a2c21850355142b41d724970d3bb1df State: starting Blocks: 0/0 BlocksInProgress: 0 Files: 0 BytesWritten: 0 CurrentVersion: -1 FirstConsistentVersion: 0 ApplyVersionLag: 0 LastError: ''Missing backup data' on 'restore_start'' 3s ago. URL: file:///tmp/backup-2024-09-26-09-35-22.992383 Range: ''-'\xff' AddPrefix: '' RemovePrefix: '' Version: 183972470321

[jzhou77]

@xiongbl526 FirstConsistentVersion: 0 says there is not a full backup present. So the problem is probably that your backup is incomplete.

[xiongbl526]

@xiongbl526 FirstConsistentVersion: 0 says there is not a full backup present. So the problem is probably that your backup is incomplete.

The issue has been resolved because I created two backup_agent pods via deployment. The backup files generated by fdbbackup were not present in the other pod. After copying the backup files over, it succeeded.