rear
"rear recover" btrfs filesytem upgrade issue with 'space_cache' mount option
Plain 'space_cache' mount option does not work as documented.
Details:
Older btrfs software supports one kind of 'space_cache'.
Newer btrfs software supports two kind of 'space_cache' namely 'space_cache=v1' and 'space_cache=v2'.
For more information see 'space_cache' at https://btrfs.readthedocs.io/en/latest/Administration.html
When "rear mkrescue" is run on a system
where its btrfs filesystem was created by older btrfs software
its mount options are shown by mount and findmnt
as ...,space_cache,... (when 'space_cache' is used)
and accordingly in disklayout.conf it appears as
fs ...,space_cache,...
...
btrfsmountedsubvol ...,space_cache,...
When the system software was upgraded to newer btrfs software but the btrfs filesystem was not upgraded to the newer btrfs default 'space_cache=v2' version then "rear mkrescue" will still list the btrfs filesystem in disklayout.conf as
fs ...,space_cache,...
...
btrfsmountedsubvol ...,space_cache,...
but "rear mkrescue" will include the newer btrfs software into the ReaR recovery system.
When then during "rear recover" the btrfs filesystem gets recreated, the newer btrfs software will recreate it with the new default 'space_cache=v2'.
In contrast to what is described in https://btrfs.readthedocs.io/en/latest/Administration.html
If a version is not explicitly specified,
the default implementation will be chosen,
which is v2.
at least in some specific newer btrfs software versions
it fails to mount a btrfs filesystem which was created
with 'space_cache=v2' when the mount option space_cache
is used where a version is not explicitly specified like
mount -t btrfs -o ...,space_cache,... /dev/... /...
It works when no 'space_cache' mount option is used
or when the mount option space_cache=v2 is used.
How to reproduce:
On SLES15-SP6 I reproduced it as follows:
Make a loop disk device:
# dd if=/dev/zero of=/loopfile bs=1024 count=262144
262144+0 records in
262144+0 records out
268435456 bytes (268 MB, 256 MiB) copied, ...
# losetup -f
/dev/loop0
# losetup /dev/loop0 /loopfile
# losetup /dev/loop0
/dev/loop0: [0034]:152685 (/loopfile)
Make a btrfs filesystem on it (empty lines removed and long lines shown wrapped here):
# mkfs -t btrfs -v /dev/loop0
btrfs-progs v6.5.1
See https://btrfs.readthedocs.io/ for more information.
Performing full device TRIM /dev/loop0 (256.00MiB) ...
NOTE: several default settings have changed in version 5.15,
please make sure this does not affect your deployments:
- DUP for metadata (-m dup)
- enabled no-holes (-O no-holes)
- enabled free-space-tree (-R free-space-tree)
Label: (null)
UUID: de15b95a-f88b-4dd5-a2e6-d6a503b98e9c
Node size: 16384
Sector size: 4096
Filesystem size: 256.00MiB
Block group profiles:
Data: single 8.00MiB
Metadata: DUP 32.00MiB
System: DUP 8.00MiB
SSD detected: yes
Zoned device: no
Incompat features: extref, skinny-metadata, no-holes,
free-space-tree
Runtime features: free-space-tree
Checksum: crc32c
Number of devices: 1
Devices:
ID SIZE PATH
1 256.00MiB /dev/loop0
Mount the btrfs filesystem with 'space_cache' mount option (long lines shown wrapped here):
# mount -t btrfs \
-o rw,relatime,ssd,discard=async,space_cache \
/dev/loop0 /loopfs
mount: /loopfs: wrong fs type, bad option,
bad superblock on /dev/loop0,
missing codepage or helper program, or other error.
dmesg(1) may have more information
after failed mount system call.
What dmesg tells about it (long lines shown wrapped here):
# dmesg | tail
...
[ 381.913886] BTRFS: device fsid
de15b95a-f88b-4dd5-a2e6-d6a503b98e9c devid 1 transid 6
/dev/loop0 scanned by mkfs.btrfs (3927)
[ 607.760390] BTRFS info (device vda2):
qgroup scan completed (inconsistency flag cleared)
[ 638.952612] BTRFS info (device loop0):
using crc32c (crc32c-intel) checksum algorithm
[ 638.952638] BTRFS info (device loop0):
enabling ssd optimizations
[ 638.952648] BTRFS info (device loop0):
turning on async discard
[ 638.952655] BTRFS info (device loop0):
enabling disk space caching
[ 638.952661] BTRFS error (device loop0):
cannot disable free space tree
[ 638.952697] BTRFS error (device loop0):
open_ctree failed
Mounting the btrfs filesystem without 'space_cache' works (long lines shown wrapped here):
# mount -t btrfs -o rw,relatime,ssd,discard=async \
/dev/loop0 /loopfs
# dmesg | tail
...
[ 1272.852179] BTRFS: device fsid
de15b95a-f88b-4dd5-a2e6-d6a503b98e9c devid 1 transid 6
/dev/loop0 scanned by mount (4394)
[ 1272.856628] BTRFS info (device loop0):
using crc32c (crc32c-intel) checksum algorithm
[ 1272.856652] BTRFS info (device loop0):
enabling ssd optimizations
[ 1272.856661] BTRFS info (device loop0):
turning on async discard
[ 1272.856667] BTRFS info (device loop0):
using free space tree
[ 1272.859103] BTRFS info (device loop0):
checking UUID tree
Also mounting the btrfs filesystem with 'space_cache=v2' works (long lines shown wrapped here):
# umount /loopfs
# mount -t btrfs \
-o rw,relatime,ssd,discard=async,space_cache=v2 \
/dev/loop0 /loopfs
# dmesg | tail
...
[ 1436.414586] BTRFS: device fsid
de15b95a-f88b-4dd5-a2e6-d6a503b98e9c devid 1 transid 8
/dev/loop0 scanned by mount (4420)
[ 1436.418086] BTRFS info (device loop0):
using crc32c (crc32c-intel) checksum algorithm
[ 1436.418103] BTRFS info (device loop0):
enabling ssd optimizations
[ 1436.418109] BTRFS info (device loop0):
turning on async discard
[ 1436.418115] BTRFS info (device loop0):
using free space tree
Solution to make "rear recover" work when there is a btrfs filesystem which was created by older btrfs software and meanwhile the system software became upgraded to newer btrfs software but the btrfs filesystem was not upgraded to the newer btrfs default 'space_cache=v2':
In etc/rear/local.conf use PRE_RECOVERY_COMMANDS
to replace in disklayout.conf ,space_cache, by ,space_cache=v2,
PRE_RECOVERY_COMMANDS=( 'sed -i -e "s/,space_cache,/,space_cache=v2,/" /var/lib/rear/layout/disklayout.conf' )
Alternatively it also works to remove space_cache,
PRE_RECOVERY_COMMANDS=( 'sed -i -e "s/,space_cache,/,/" /var/lib/rear/layout/disklayout.conf' )
Such PRE_RECOVERY_COMMANDS are not needed
when a btrfs filesystem was created by newer btrfs software
with its newer btrfs default 'space_cache=v2' because then
its mount options are shown by mount and findmnt
as ...,space_cache=v2,... (when 'space_cache' is used)
and accordingly in disklayout.conf it appears as
fs ...,space_cache=v2,...
...
btrfsmountedsubvol ...,space_cache=v2,...
where things work.