kdave
mkfs: create initial subvolume
Create an initial subvolume or subvolumes. Usecase:
- allow --root-dir to put the files there
- allow creating nested subvolumes
- allow creating arbitrary paths
- allow multiple subvolumes
https://patchwork.kernel.org/patch/9930323/ + other patches in the mailingist
Initial support added to devel, --rootdir + --subvol will fill the given subvolume. The final semantics will be something like this:
Sounds good, I think we can achieve more than just subvol + copy of a
directory, with a extension of the command line ui. This should allow to
create multiple subvolumes, nested (or not) with contents of the
directory copied over any pre-created contents from --rootdir.
First, the option --subvol could be specified repeatedly. Each path
component that would not exist will become a directory, the last one
will be subvolume.
With --rootdir among the --subvol options, all existing paths would be
reused (as long files and directories/subvolumes don't clash).
Example, immitating one of used root filesystem layout:
expected result
/
/@ (subvolume
/@/boot (subvolume)
/@/home (subvolume)
/@/var/log (subvolume)
/@/tmp (subvolume)
and over that, same as if some tarball is unpacked, creating /bin,
/etc, /usr, /sbin etc.
would be generated by by something like:
$ mkfs.btrfs -S /@ -S /@/boot -S /@/home -S /@/var/log -S /@/tmp --rootdir rootfiles/ /path/to/device
Reusing --rootdir might be a bad idea, so we'd have add a new option
that would specify the target path and the source directory.
For the --rootdir usage with --subvolume, we can add an extra <dst>: prefix, to specify which subvolume the rootdir would be utilized.
For now, I'd prefer to implement a single --subvolume option as a first step, then the combination with --rootdir, and finally repeated --subvolume along with repeated --rootdir.
@adam900710 My idea was that you'd just pass directories in --rootdir that should be subvolumes in the generated filesystem. So if I do --subvolume /usr then /usr from the user provided root directory will be a subvolume in the generated filesystem.
Specifically, I don't want to specify into which subvolume --rootdir should be stored. Instead I want to specify which directories in --rootdir should become subvolumes.
I don't think btrfs-progs itself needs to implement repeated --rootdir. Think of it this way, usually applications that are creating filesystems and images aren't using just mkfs.btrfs. They might also support mkfs.ext4, mkfs.xfs and other filesystems. Even if mkfs.btrfs supports repeated --rootdir, it won't be supported by the other mkfs tools, so the application has to implement the repeated --rootdir functionality itself anyway (by merging all the repeated rootdirs into one single rootdir before calling mkfs). On the other hand, --subvolume is something that applications cannot implement themselves (unless they start using loop devices and mounting the filesystem), so this is where we need help from mkfs.btrfs.
So I wouldn't focus on repeated --rootdir as applications can handle that themselves, I would focus on --subvolume as this is the btrfs specific feature that applications cannot implement themselves without using loop devices and mount.
My idea was that you'd just pass directories in --rootdir that should be subvolumes in the generated filesystem. So if I do --subvolume /usr then /usr from the user provided root directory will be a subvolume in the generated filesystem.
This is very reasonable.
Just with some corner cases, like what if the the subvolume provided doesn't exist inside the --rootdir option (creating one is easy), but what if the subvolume name conflicts with one non-directory file? (Error out AFAIK).
I believe those corner cases can be solved without much hassle though.
The challenge may come from the the extra subvolume lookup needed to copy the files, and the sequence between --rootdir copying and --subvolume creation.
This may be a little more complex than I initially thought.
Thus for the first step, I'm afraid we have to make --rootdir and --subvolume exclusive, meaning if --rootdir is specified, no --subvolume is allowed, and vice verse, to make us focus on the new --subvolume option.
Then we can determine which way to go. Although I believe the single --rootdir (and always at top level subvolume) with multiple --subvolume options is the best solution so far.
Just with some corner cases, like what if the the subvolume provided doesn't exist inside the --rootdir option (creating one is easy), but what if the subvolume name conflicts with one non-directory file? (Error out AFAIK).
I would do an initial pass to check that all given subvolumes exist as directories in the given root directory and error out if they don't. --subvolume seems like it should solely be about indicating existing directories that should become subvolumes, not creating new directories.
The challenge may come from the the extra subvolume lookup needed to copy the files, and the sequence between --rootdir copying and --subvolume creation. This may be a little more complex than I initially thought.
When I looked at implementing myself I was thinking that these would be done together in traverse_directory(). First, before calling traverse_directory(), collect a list of inodes corresponding to the directories in --rootdir that should become subvolumes. Then, in traverse_directory(), whenever we encounter an inode that should become a subvolume (by checking the list), do the magic to make it a subvolume (this was where I got stuck) and then call traverse_directory() (or maybe btrfs_mkfs_fill_dir()) recursively from traverse_directory() to populate the subvolume. By calling traverse_directory() recursively for each subvolume we wouldn't have to keep track of which file maps to which subvolume in traverse_directory() itself.
I would do an initial pass to check that all given subvolumes exist as directories in the given root directory and error out if they don't.
I'd say it's a little too strict, maybe the user just really want to create a subvolume if not included in the source directory.
Especially considering --subvolume can be specified without --rootdir option.
When I looked at implementing myself I was thinking that these would be done together in traverse_directory(). ...
I tend to do it in another way, firstly create all the involved subvolumes (along with their parent directories), then let traverse_directory() to do the regular copies.
The difference would be, when creating directories, we need to check if the directory exists.
If exists and it's a directory, just skip the directory creation and continue copying. (I'm already adding those checks to the existing infrastructures)
If it's a subvolume, we need to copy the children into the subvolume, which may need some changes to the code.
Anyway I will try to create a PoC in recent days for this --subvolume idea (without --rootdir support as the first step though)