grub-btrfs
grub-btrfs copied to clipboard
Antynea
GRUB snapshots not updating after rollback
Hi there, it's me from #183 . I've made significant progress thanks to a Calamares patch that lets me install it without a @ subvolume for /. Now my fstab looks like this after installation:
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=c41d9810-bc7c-45b8-a9dc-2a1a1cb8806e / btrfs subvol=,defaults,noatime,space_cache,autodefrag,compress=zstd 0 0
UUID=c41d9810-bc7c-45b8-a9dc-2a1a1cb8806e /.snapshots btrfs subvol=/@snapshots,defaults,noatime,space_cache,autodefrag,compress=zstd 0 0
UUID=c41d9810-bc7c-45b8-a9dc-2a1a1cb8806e /home btrfs subvol=/@home,defaults,noatime,space_cache,autodefrag,compress=zstd 0 0
UUID=c41d9810-bc7c-45b8-a9dc-2a1a1cb8806e /var btrfs subvol=/@var,defaults,noatime,space_cache,autodefrag,compress=zstd 0 0
And btrfs sub list / immediately after installation:
ID 257 gen 9 top level 5 path @snapshots
ID 258 gen 110 top level 5 path @home
ID 259 gen 110 top level 5 path @var
ID 266 gen 104 top level 5 path .snapshots
ID 267 gen 104 top level 258 path @home/.snapshots
I'm also now putting /var in a separate subvolume, which allows for booting RO snapper snapshots all the way into LightDM and from there into Mate. From there I'm running snapper --ambit classic rollback, which correctly rolls it back to whatever snapshot I booted into and sets that as default from then on. However, from then on future attempts to run grub2-mkconfig -o /boot/grub2/grub.cfg detects the newer snapshots and it updates /boot/grub2/grub-btrfs.cfg, but the newer snapshots don't appear in my GRUB menu. I gather that this is similar to #154 and #155 , that GRUB is booting from an older snapshot of /boot but I'm not sure why. Here's what my subvolumes look like after a few rollback attempts:
sam-virtualbox:/home/sam # btrfs sub list /
ID 257 gen 212 top level 5 path @snapshots
ID 258 gen 225 top level 5 path @home
ID 259 gen 227 top level 5 path @var
ID 266 gen 104 top level 5 path .snapshots
ID 267 gen 149 top level 258 path @home/.snapshots
ID 268 gen 116 top level 257 path @snapshots/1/snapshot
ID 269 gen 118 top level 257 path @snapshots/2/snapshot
ID 270 gen 125 top level 257 path @snapshots/3/snapshot
ID 271 gen 127 top level 257 path @snapshots/4/snapshot
ID 272 gen 128 top level 257 path @snapshots/5/snapshot
ID 273 gen 134 top level 257 path @snapshots/6/snapshot
ID 274 gen 137 top level 257 path @snapshots/7/snapshot
ID 275 gen 144 top level 257 path @snapshots/8/snapshot
ID 276 gen 145 top level 257 path @snapshots/9/snapshot
ID 277 gen 198 top level 257 path @snapshots/10/snapshot
ID 278 gen 149 top level 267 path @home/.snapshots/1/snapshot
ID 279 gen 211 top level 257 path @snapshots/11/snapshot
ID 280 gen 153 top level 257 path @snapshots/12/snapshot
ID 281 gen 171 top level 257 path @snapshots/13/snapshot
ID 282 gen 200 top level 257 path @snapshots/14/snapshot
ID 283 gen 179 top level 257 path @snapshots/15/snapshot
ID 284 gen 184 top level 257 path @snapshots/16/snapshot
ID 285 gen 197 top level 257 path @snapshots/17/snapshot
ID 286 gen 213 top level 257 path @snapshots/18/snapshot
ID 287 gen 210 top level 257 path @snapshots/19/snapshot
ID 288 gen 224 top level 257 path @snapshots/20/snapshot
sam-virtualbox:/home/sam # snapper list
# | Type | Pre # | Date | User | Cleanup | Description | Userdata
----+--------+-------+--------------------------+------+---------+------------------------+--------------
0 | single | | | root | | current |
1 | pre | | wed 01 dec 2021 19:47:21 | root | number | zypp(zypper) | important=no
2 | post | 1 | wed 01 dec 2021 19:47:24 | root | number | | important=no
3 | pre | | wed 01 dec 2021 19:51:36 | root | number | yast snapper |
4 | single | | wed 01 dec 2021 19:52:23 | root | | grub-btrfs ready |
5 | post | 3 | wed 01 dec 2021 19:52:26 | root | number | |
6 | pre | | wed 01 dec 2021 19:56:22 | root | number | zypp(zypper) | important=no
7 | post | 6 | wed 01 dec 2021 19:56:23 | root | number | | important=no
8 | pre | | wed 01 dec 2021 19:59:32 | root | number | yast snapper |
9 | single | | wed 01 dec 2021 19:59:49 | root | | grub-btrfs generated |
10 | post | 8 | wed 01 dec 2021 19:59:52 | root | number | |
11 | pre | | wed 01 dec 2021 20:01:05 | root | number | zypp(zypper) | important=no
12 | post | 11 | wed 01 dec 2021 20:01:06 | root | number | | important=no
13 | single | | wed 01 dec 2021 20:07:40 | root | number | rollback backup | important=yes
14 | single | | wed 01 dec 2021 20:07:41 | root | number | writable copy of #10 |
15 | pre | | wed 01 dec 2021 20:10:13 | root | number | zypp(zypper) | important=no
16 | post | 15 | wed 01 dec 2021 20:10:23 | root | number | | important=no
17 | single | | wed 01 dec 2021 20:20:52 | root | number | rollback backup of #14 | important=yes
18 | single | | wed 01 dec 2021 20:20:53 | root | number | writable copy of #10 |
19 | single | | wed 01 dec 2021 20:24:30 | root | number | rollback backup of #18 | important=yes
20* | single | | wed 01 dec 2021 20:24:30 | root | | writable copy of #11 |
Any ideas? Thanks a lot!
Sorry for the dumb report... Turns out that with openSUSE you need to run update-bootloader (or run the YaST Bootloader tool) to make it actually update GRUB.
No problem, and thanks for this project! It's really an excellent feature for Linux on Btrfs installations that was formerly only available with vanilla openSUSE and otherwise very hard to implement.
Hi again @Antynea , it looks like update-bootloader doesn't fix the problem. It appears that only running the YaST Bootloader GUI correctly updates the list of snapshots in GRUB itself, although the .cfg files are being properly generated most of the time (I'm using your instructions for systemd to update the config automatically whenever Snapper creates an automatic snapshot, for example when adding/removing packages with the zypper command. But I also tried manually running grub2-mkconfig -o /boot/grub2/grub.cfg and update-bootloader, it usually seems to make no difference). After running YaST Bootloader and installing it to /dev/sda it seems to work automatically again, but future snapper rollback operations can make it stop updating the list again. I'm sending you a complete history of a fresh install from an ISO I created with grub-btrfs pre-configured. Please let me know if you need anything else. I'm sorry that this is really seeming kind of random and I'm quite confused. Thanks a lot for the help!
0.txt
1.txt
After rollback to Snapshot 1...
2.txt
3.txt
4.txt
5.txt
6.txt
After rollback to Snapshot 11...
7.txt
I was also able to reproduce this same behavior on my Thinkpad T440s, so apparently it's not some sort of virtualizer quirk.
hello,
I don't understand what yast2 bootloader does that is not already done with update-bootlaoder + grub2-mkconfig.
update-bootlaoder reinstall Grub.
grub2-mkconfig update grub.cfg file
Hi, thanks a lot for the reply. I had the same question. I dug through the YaST Ruby code a bit, but mainly it can change options in /etc/default/grub and /etc/sysconfig/bootloader and then runs update-bootloader (which in turn runs grub2-mkconfig) with a bit more logic based on whether some /etc/grub.d scripts and environment variables are present. I thought maybe one of the options in /etc/default/grub or /etc/sysconfig/bootloader was making a difference, but as far as it relates to btrfs-grub I've tried all the combinations I can think of and it doesn't change anything.
Would you be interested in trying my test ISO?
Would you be interested in trying my test ISO?
Sure, but during the holiday season, I'm not very available. It's going to be difficult to get more involved until next year :)
I understand, I appreciate your time! Meanwhile any other tips perchance? Is there anything fundamentally weird with my /etc/fstab or my subvolume layout or something that Snapper is doing maybe?
Hi again @Antynea , I think I finally made some significant progress with this weird issue. I don't understand why exactly, but I discovered that if I manually edit the kernel command line in GRUB from:
/boot/vmlinuz-5.15.6-1-default root=UUID=4887f89a-a1b5-47e4-8fe7-8f1e9c46fb19 quiet splash
to
/boot/vmlinuz-5.15.6-1-default quiet splash
(eliminating the root= parameter) then after doing a snapper rollback and grub2-mkconfig -o /boot/grub2/grub.cfg GRUB continues to be updated.
If I don't make that change then GRUB won't update at all after doing a rollback, even if I change my GRUB_DISTRIBUTOR value or other core GRUB settings. And even if I completely remove all traces of grub-btrfs from the system GRUB still remains stuck where it was before the rollback and won't update (although the /boot/grub2/ .cfg files are correctly updated) unless I run grub2-install /dev/sda. (I imagine the reason that YaST Bootloader also is able to fix the problem means that it also runs grub2-install at some point.) So this led me to suspect that GRUB is looking for its .cfg file(s) on the wrong subvolume. This post appears to confirm that is what is happening, although I still don't understand why:
https://bbs.archlinux.org/viewtopic.php?pid=1615360#p1615360
So what I did is roughly imitate the way that openSUSE does its GRUB integration with Snapper snapshots (I can't use openSUSE's native method because their script has a bug that breaks dual-boot configurations). I discovered that they use two files:
- /etc/grub.d/80_suse_btrfs_snapshot
- /usr/lib/snapper/plugins/grub
What they do is instead of putting their
grub-snapshot.cfgfile in/boot/grub2/they put it in/.snapshots/, and then that in turn makes a list of all the/.snapshots/XXX/grub-snapshot.cfgthat exist. So it appears that by putting it on the/.snapshotssubvolume, not on the main root of the Btrfs filesystem (which for various reasons I need to not be a@subvolume like most distros do) it prevents the GRUB installation from looking into the wrong subvolume for its .cfg files.
I finally arrived at this workaround by hacking your scripts with some hard-coded references to /.snapshots as follows:
Do you think that you could include some config variables to make this sort of configuration possible, or should I stick to my hacked version?
Thanks a lot for your time and help!
Hi, what you did there in the 41_snaphshots-btrfs script is basically exchanging grub_directory with /.snapshots. Is this correct?
You could just set the variable GRUB_BTRFS_GRUB_DIRNAME in /etc/default/grub-btrfs/config for that.
I know the comment ## Customize GRUB directory, where "grub.cfg" file is saved is a bit confusing.
There is one place where you used /.snapshots where you exchanged something different than $grub_direcory which is this line:
configfile "/.snapshots/grub-btrfs.cfg"
The original:
configfile "\${prefix}/grub-btrfs.cfg"
I will look into not using ${prefix} here and using the configurable $grub_directory variable here as well.
I still don't think this is the problem you have. I think you are taking snapshots of /boot which you roll back therefore rolling back the grub-btrfs.cfg as well or something. That is why the location of the grub-btrfs.cfg file matters.
Hi @Schievel1 thanks a lot for the reply.
I still don't think this is the problem you have. I think you are taking snapshots of /boot which you roll back therefore rolling back the grub-btrfs.cfg as well or something.
I agree that this isn't related to the fundamental problem of the GRUB snapshots menu not updating after a rollback and requiring a grub-install .... to revive it. This appears to be the explanation:
https://bbs.archlinux.org/viewtopic.php?pid=1615360#p1615360
It appears that GRUB is looking for the grub-btrfs.cfg in the last snapshot subvolume before the rollback, and then it starts working again after running grub-install .... because that makes GRUB look for it on the current root subvol.
Hm I don't really know how I could fix that from grub-btrfs. This seems to be an issue of grub and how snapshots are setup on the system.
However I will still try to make the ${prefix} configurable so it will be possible to have grub-btrfs.cfg in a different directory then where grub is.
However I will still try to make the ${prefix} configurable so it will be possible to have grub-btrfs.cfg in a different directory then where grub is.
Yes, that would still be useful, thanks!
Hey, I just pushed a branch where prefix is configurable. Look into config or manpage, you basically can set GRUB_BTRFS_GBTRFS_SEARCH_DIRNAME for where the file grub-btrfs.cfg should be stored and GRUB_BTRFS_GBTRFS_SEARCH_DIRNAME for where grub is looking for it. Those should both be set to /.snapshots for you, I think.
Please be aware, for this to take effect you need to run grub-mkconfig (or whatever is your distros equivalent) not just /etc/grub.d/41_snapshots-btrfs.
Uhm yeah, does it work?
It would require a lot of testing and reworking the current GRUB snapshots implementation that I'm using in the current version of GeckoLinux, so I'll get back with you the next time I make a release. (Current GeckoLinux uses the default openSUSE GRUB scripts with a small hack I added. Whereas SpiralLinux is working well with the previous version of grub-btrfs by default.) I think that the btrfs-grub project has done as much as possible to allow for a working solution, so thanks a lot for that. From there the user or the distributor will have to figure out the exact implementation details.
Oh so geckolinux is not just your Nick but the name of a distribution? good to know. I will merge the PR then, let me know if you run into any issues when you test it out.
Oh so geckolinux is not just your Nick but the name of a distribution?
Yep, I'm the maintainer of GeckoLinux (built from openSUSE) and SpiralLinux (built from Debian Stable). When I opened this bug report I was working on GeckoLinux and struggling to setup openSUSE's native GRUB snapshot scripts with the Calamares installer (instead of openSUSE's native YaST installer). So I tried grub-btrfs, but since I couldn't workaround the issue in this bug report I ended up going back to openSUSE's native GRUB snapshots and finally found a workaround. On the other hand SpiralLinux does successfully use grub-btrfs and ships it as the default.