baksnapper icon indicating copy to clipboard operation
baksnapper copied to clipboard

Baksnapper backing up snapshot with wrong parent

Open Gooberpatrol66 opened this issue 11 months ago • 7 comments

It's backing up 96 with 63 as the parent

+ break
+ incremental-backup 62 63
+ echo 'Incremental backup'
Incremental backup
+ ssh -i /root/.ssh/backup-user-key [email protected] create-snapshot /media/offsite-backup/nathan-guixlaptop 63
+ baksnapperd send-info /home/nathan/.snapshots 63
+ ssh -i /root/.ssh/backup-user-key [email protected] receive-info /media/offsite-backup/nathan-guixlaptop 63
+ baksnapperd send-incremental-snapshot /home/nathan/.snapshots/62 /home/nathan/.snapshots/63
+ ssh -i /root/.ssh/backup-user-key [email protected] receive-snapshot /media/offsite-backup/nathan-guixlaptop 63
At subvol /home/nathan/.snapshots/63/snapshot
At snapshot snapshot
+ [[ 1 -eq 1 ]]
+ ssh -i /root/.ssh/backup-user-key [email protected] link-latest /media/offsite-backup/nathan-guixlaptop
+ (( idx=1 ))
+ (( idx < num_src_only ))
+ incremental-backup 63 96
+ echo 'Incremental backup'
Incremental backup
+ ssh -i /root/.ssh/backup-user-key [email protected] create-snapshot /media/offsite-backup/nathan-guixlaptop 96
+ baksnapperd send-info /home/nathan/.snapshots 96
+ ssh -i /root/.ssh/backup-user-key [email protected] receive-info /media/offsite-backup/nathan-guixlaptop 96
kex_exchange_identification: read: Connection reset by peer
Connection reset by 10.9.0.4 port 22
+ ssh -i /root/.ssh/backup-user-key [email protected] remove-broken-snapshot /media/offsite-backup/nathan-guixlaptop 63
Delete subvolume (no-commit): '/media/offsite-backup/nathan-guixlaptop/63/snapshot'
+ error 'Failed to send snapshot info!'
+ echo '[ERROR] Failed to send snapshot info!'
[ERROR] Failed to send snapshot info!
+ exit 1

snapshot 96 is not a direct child of snapshot 63

 # │ Type   │ Pre # │ Date                            │ User │ Cleanup  │ Description │ Userdata
───┼────────┼───────┼─────────────────────────────────┼──────┼──────────┼─────────────┼─────────
 0 │ single │       │                                 │ root │          │ current     │
 1 │ single │       │ Sun 24 Jul 2022 05:36:34 PM CDT │ root │ timeline │             │
 2 │ single │       │ Mon 25 Jul 2022 08:26:26 AM CDT │ root │ timeline │             │
 3 │ single │       │ Sat 06 Aug 2022 03:43:43 PM CDT │ root │ timeline │             │
 4 │ single │       │ Tue 30 Aug 2022 05:47:54 PM CDT │ root │ timeline │             │
 5 │ single │       │ Thu 01 Sep 2022 05:04:35 PM CDT │ root │ timeline │             │
 6 │ single │       │ Thu 01 Sep 2022 05:06:57 PM CDT │ root │ timeline │             │
 7 │ single │       │ Sat 03 Sep 2022 06:03:13 PM CDT │ root │ timeline │             │
 8 │ single │       │ Sun 25 Sep 2022 11:49:52 PM CDT │ root │ timeline │             │
 9 │ single │       │ Mon 26 Sep 2022 05:55:17 PM CDT │ root │ timeline │             │
10 │ single │       │ Mon 26 Sep 2022 06:51:39 PM CDT │ root │ timeline │             │
11 │ single │       │ Tue 04 Oct 2022 05:50:39 PM CDT │ root │ timeline │             │
12 │ single │       │ Tue 11 Oct 2022 05:51:18 PM CDT │ root │ timeline │             │
13 │ single │       │ Tue 11 Oct 2022 06:14:44 PM CDT │ root │ timeline │             │
14 │ single │       │ Tue 11 Oct 2022 06:16:37 PM CDT │ root │ timeline │             │
15 │ single │       │ Sun 16 Oct 2022 06:29:58 PM CDT │ root │ timeline │             │
16 │ single │       │ Sun 16 Oct 2022 06:30:21 PM CDT │ root │ timeline │             │
17 │ single │       │ Tue 18 Oct 2022 06:31:50 PM CDT │ root │ timeline │             │
18 │ single │       │ Thu 20 Oct 2022 04:33:51 PM CDT │ root │ timeline │             │
19 │ single │       │ Mon 24 Oct 2022 09:02:48 PM CDT │ root │ timeline │             │
20 │ single │       │ Mon 24 Oct 2022 09:18:35 PM CDT │ root │ timeline │             │
21 │ single │       │ Fri 28 Oct 2022 07:54:20 AM CDT │ root │ timeline │             │
22 │ single │       │ Sat 29 Oct 2022 04:31:16 PM CDT │ root │ timeline │             │
23 │ single │       │ Wed 02 Nov 2022 04:57:02 PM CDT │ root │ timeline │             │
24 │ single │       │ Thu 03 Nov 2022 07:01:41 PM CDT │ root │ timeline │             │
25 │ single │       │ Mon 07 Nov 2022 05:17:08 PM CST │ root │ timeline │             │
26 │ single │       │ Tue 08 Nov 2022 06:41:48 PM CST │ root │ timeline │             │
27 │ single │       │ Thu 01 Dec 2022 06:57:09 PM CST │ root │ timeline │             │
28 │ single │       │ Wed 04 Jan 2023 04:40:39 PM CST │ root │ timeline │             │
29 │ single │       │ Fri 20 Jan 2023 03:33:12 PM CST │ root │ timeline │             │
30 │ single │       │ Sun 05 Feb 2023 04:54:20 PM CST │ root │ timeline │             │
31 │ single │       │ Mon 06 Feb 2023 06:00:19 PM CST │ root │ timeline │             │
32 │ single │       │ Sat 18 Feb 2023 05:29:42 PM CST │ root │ timeline │             │
33 │ single │       │ Mon 27 Mar 2023 03:31:23 PM CDT │ root │ timeline │             │
34 │ single │       │ Mon 27 Mar 2023 03:32:11 PM CDT │ root │ timeline │             │
35 │ single │       │ Tue 04 Apr 2023 06:34:30 PM CDT │ root │ timeline │             │
36 │ single │       │ Thu 20 Apr 2023 07:41:20 PM CDT │ root │ timeline │             │
37 │ single │       │ Fri 21 Apr 2023 04:57:29 PM CDT │ root │ timeline │             │
38 │ single │       │ Mon 15 May 2023 05:07:21 PM CDT │ root │ timeline │             │
39 │ single │       │ Mon 15 May 2023 05:07:53 PM CDT │ root │ timeline │             │
40 │ single │       │ Mon 26 Jun 2023 06:00:24 PM CDT │ root │ timeline │             │
41 │ single │       │ Sat 08 Jul 2023 08:36:06 PM CDT │ root │ timeline │             │
42 │ single │       │ Sun 09 Jul 2023 02:09:20 PM CDT │ root │ timeline │             │
43 │ single │       │ Thu 27 Jul 2023 10:20:16 PM CDT │ root │ timeline │             │
44 │ single │       │ Sat 05 Aug 2023 03:15:23 PM CDT │ root │ timeline │             │
46 │ single │       │ Tue 15 Aug 2023 05:52:43 PM CDT │ root │ timeline │             │
47 │ single │       │ Thu 14 Sep 2023 02:44:28 PM CDT │ root │ timeline │             │
48 │ single │       │ Sun 24 Sep 2023 06:31:17 PM CDT │ root │ timeline │             │
49 │ single │       │ Fri 29 Sep 2023 03:08:49 PM CDT │ root │ timeline │             │
50 │ single │       │ Sun 01 Oct 2023 03:58:04 PM CDT │ root │ timeline │             │
51 │ single │       │ Wed 25 Oct 2023 01:09:05 AM CDT │ root │ timeline │             │
52 │ single │       │ Wed 25 Oct 2023 06:15:48 PM CDT │ root │ timeline │             │
53 │ single │       │ Mon 30 Oct 2023 04:22:57 PM CDT │ root │ timeline │             │
55 │ single │       │ Mon 06 Nov 2023 03:10:40 PM CST │ root │ timeline │             │
57 │ single │       │ Mon 27 Nov 2023 06:44:47 PM CST │ root │ timeline │             │
58 │ single │       │ Fri 01 Dec 2023 03:20:57 PM CST │ root │ timeline │             │
60 │ single │       │ Tue 09 Jan 2024 07:28:23 PM CST │ root │ timeline │             │
61 │ single │       │ Fri 19 Jan 2024 05:24:15 PM CST │ root │ timeline │             │
62 │ single │       │ Tue 23 Jan 2024 06:14:53 PM CST │ root │ timeline │             │
63 │ single │       │ Wed 14 Feb 2024 07:30:35 PM CST │ root │ timeline │             │
64 │ single │       │ Wed 21 Feb 2024 08:21:29 PM CST │ root │ timeline │             │
65 │ single │       │ Tue 19 Mar 2024 04:42:06 PM CDT │ root │ timeline │             │
66 │ single │       │ Thu 21 Mar 2024 05:50:34 PM CDT │ root │ timeline │             │
73 │ single │       │ Fri 12 Apr 2024 03:42:56 AM CDT │ root │ timeline │             │
75 │ single │       │ Wed 17 Apr 2024 01:59:48 PM CDT │ root │ timeline │             │
76 │ single │       │ Sun 05 May 2024 06:53:06 PM CDT │ root │ timeline │             │
77 │ single │       │ Wed 15 May 2024 03:27:29 PM CDT │ root │ timeline │             │
78 │ single │       │ Thu 16 May 2024 12:23:07 AM CDT │ root │ timeline │             │
79 │ single │       │ Sun 26 May 2024 08:29:30 PM CDT │ root │ timeline │             │
80 │ single │       │ Wed 29 May 2024 12:16:24 AM CDT │ root │ timeline │             │
81 │ single │       │ Wed 29 May 2024 02:25:36 AM CDT │ root │ timeline │             │
82 │ single │       │ Thu 04 Jul 2024 07:00:47 PM CDT │ root │ timeline │             │
83 │ single │       │ Sat 20 Jul 2024 08:04:24 AM CDT │ root │ timeline │             │
84 │ single │       │ Wed 31 Jul 2024 08:42:15 PM CDT │ root │ timeline │             │
85 │ single │       │ Sat 03 Aug 2024 08:12:55 PM CDT │ root │ timeline │             │
86 │ single │       │ Thu 08 Aug 2024 05:52:33 PM CDT │ root │ timeline │             │
87 │ single │       │ Fri 09 Aug 2024 04:18:07 PM CDT │ root │ timeline │             │
88 │ single │       │ Fri 09 Aug 2024 05:43:45 PM CDT │ root │ timeline │             │
89 │ single │       │ Fri 09 Aug 2024 05:54:10 PM CDT │ root │ timeline │             │
90 │ single │       │ Fri 09 Aug 2024 06:13:45 PM CDT │ root │ timeline │             │
91 │ single │       │ Tue 13 Aug 2024 04:10:25 PM CDT │ root │ timeline │             │
92 │ single │       │ Tue 03 Sep 2024 06:02:36 PM CDT │ root │ timeline │             │
93 │ single │       │ Fri 27 Sep 2024 04:21:49 PM CDT │ root │ timeline │             │
94 │ single │       │ Sun 27 Oct 2024 09:14:28 PM CDT │ root │ timeline │             │
95 │ single │       │ Thu 05 Dec 2024 06:32:13 PM CST │ root │ timeline │             │
96 │ single │       │ Sat 21 Dec 2024 06:55:17 PM CST │ root │ timeline │             │
97 │ single │       │ Tue 31 Dec 2024 11:40:47 PM CST │ root │ timeline │             │
98 │ single │       │ Mon 13 Jan 2025 11:44:49 PM CST │ root │ timeline │             │
99 │ single │       │ Mon 27 Jan 2025 05:44:04 PM CST │ root │ timeline │             │

Gooberpatrol66 avatar Jan 28 '25 21:01 Gooberpatrol66

What snapshots do you have at the backup location? The logic for it should be to use the latest snapshot at the backup location to send an incremental snapshot. So technically nothing wrong with it using 63 as the base. But it did send 63 first which seems odd.

Looks like the interrupt logic needs some polishing.

Can you post the full verbose log?

What version of baksnapper and baksnapperd are you using?

Note: I won't be able to take a proper look at this for a few weeks.

plattfot avatar Jan 29 '25 19:01 plattfot

The logic for it should be to use the latest snapshot at the backup location to send an incremental snapshot. So technically nothing wrong with it using 63 as the base.

Yeah but it uses a lot more storage and takes a lot longer than using 95 as the base

Can you post the full verbose log?

log.txt

What version of baksnapper and baksnapperd are you using?

2.3.0

Gooberpatrol66 avatar Feb 04 '25 06:02 Gooberpatrol66

The logic for it should be to use the latest snapshot at the backup location to send an incremental snapshot. So technically nothing wrong with it using 63 as the base.

Yeah but it uses a lot more storage and takes a lot longer than using 95 as the base

Indeed. Not great.

Can you post the full verbose log?

log.txt

What version of baksnapper and baksnapperd are you using?

2.3.0

Thanks!

I should have some time to work on this next week. I'll keep you posted.

plattfot avatar Feb 04 '25 10:02 plattfot

Ok, I think I know what's going on. It will require some re-engineering on how it gather and compares the snapshots between the two sites. It will take some time to implement this. I'll keep you posted on my progress.

plattfot avatar Feb 11 '25 19:02 plattfot

Apologize, life got in the way so I haven't have time to work on this until now. But I should have some time for it going forward. It will take some time to implement this given that this affect one of the core algorithms. For that I need to create some unit test to make sure I don't break anything. And for that I need a better build system.

plattfot avatar Apr 18 '25 23:04 plattfot

Hello, what's the status of this? My backup drives have filled up due to this bug.

Gooberpatrol66 avatar May 29 '25 19:05 Gooberpatrol66

Hi, Things are slowly going forward. I'm still working on writing a test runner. You can follow the progress in this branch. Having to write a test framework plus life threw my I curve ball so I have less time than I was hoping for are the main reasons it is taking so long. Hard to say how long it will take to finish. The test runner is almost complete, but always when you think you are close to the end it turns out there are more things just around the corner. But I'll try to be more active on this issue to report the progress.

Apologize for your backup filled up, a filled up btrfs drive is a pain to deal with.

I did discover that snapper does have a built in functionality for sending backups — it didn't have that when I started this project. It's called snbk. I haven't tried it myself but maybe worth testing since you got a blocking bug in Baksnapper.

plattfot avatar May 30 '25 01:05 plattfot

Think I got the test runner in a somewhat working state #29. I just need to add some more tests — one of them being a repro for this issue. Once I have that I can start fixing this bug!

plattfot avatar Jun 01 '25 03:06 plattfot

Ok, got the test runner in a good state. I'll start setting up a test to try and reproduce the issue.

plattfot avatar Jun 05 '25 02:06 plattfot

And I now got a test that reproduces the issue.

plattfot avatar Jun 05 '25 03:06 plattfot

Alright, got a fix for it. Could you test it before I merge it? It passes all my tests — including the repro one — but I don't have full coverage yet.

plattfot avatar Jun 05 '25 05:06 plattfot

I backed up and didn't see any wrong parents, so I suppose it's fixed. Thank you!

Gooberpatrol66 avatar Jun 09 '25 23:06 Gooberpatrol66

Thanks for verifying. And sorry for it taking such a long time to fix it. Now with the test runner in place, future bugs/features should be faster to implement.

plattfot avatar Jun 10 '25 02:06 plattfot

This is now included in the 2.4.1 release.

plattfot avatar Jun 10 '25 02:06 plattfot