bootc switch `--stateroot` flag

[omertuc]

Background

added the stateroot option to the install subcommand

Issue

The stateroot option is not available on the switch subcommand

Solution

Add the stateroot option to the switch subcommand

Implementation

• If the stateroot is different than the current, we should allow using the same image as the currently booted one

• Stateroot has to be explicitly created (init_osname binding) if it doesn't exist. If it does, we still call init_osname and simply ignore the error (TODO: only ignore non-already-exists errors)

• Copy /var from the old stateroot to the new one. I'm doing --reflink but it's still very slow

• Must use the old stateroot to find the merge_deployment because otherwise we boot without the required kargs (it manifested as a missing root=UUID=... which caused the dracut rootfs-generator to silently fail to create sysroot.mount and so ostree-prepare-root failed due to empty /sysroot)

[omertuc]

After shelling into the podman-bootc VM and running:

ostree admin stateroot-init foo
bootc switch --stateroot foo quay.io/fedora/fedora-bootc:40

I've tried to then reboot (simply reboot in the shell) into the new stateroot, but the only grub option is the original CentOS stateroot.

I've also noticed that after rebooting the machine, podman-bootc list shows the machine is not running, so I have to podman-bootc run <machine-id> to get it to run.

I've also observed the following weird behavior:

Ran the above commands and then:

[root@localhost ~]# ostree admin status
  foo 51d7dcab77ab6a4e7235bbdbbe32f70f6f97b6a679f7a359bfea346ae2f18b0e.0 (staged)
    origin: <unknown origin type>
* default f96c05656ae82d9758be08bcf888d8f130fafaf9723967255bd76b51f26bd2e4.0
    origin: <unknown origin type>
[root@localhost ~]# ostree admin set-default 0
  foo 51d7dcab77ab6a4e7235bbdbbe32f70f6f97b6a679f7a359bfea346ae2f18b0e.0 (staged)
    origin: <unknown origin type>
* default f96c05656ae82d9758be08bcf888d8f130fafaf9723967255bd76b51f26bd2e4.0
    origin: <unknown origin type>
[root@localhost ~]# ostree admin set-default 1
[root@localhost ~]# ostree admin status
* default f96c05656ae82d9758be08bcf888d8f130fafaf9723967255bd76b51f26bd2e4.0
    origin: <unknown origin type>

The stateroot just disappears? That's a bit odd

[cgwalters]

Let's split the install vs switch changes into separate PRs; the install one I think we can trivially land with just a simple integration test.

[omertuc]

Split install CLI into https://github.com/containers/bootc/pull/622

[omertuc]

Should we provide users with a CLI option to skip the /var copy? Or maybe skipping should be the default?

[cgwalters]

This one heavily relates to https://github.com/containers/bootc/issues/404

[cgwalters]

I think my overall inclination here is to actually make this part of bootc install, and roll up fixing things related to https://github.com/containers/bootc/pull/137 at the same time.

Hence because it's install I think I'd argue we drop all state by default...except let's add a check for "you have root in current kargs but missing in new kargs". That would avoid probably the biggest footgun and help people understand that often they're going to need to allow-list in at least some state from the current root.

Something like bootc install --stateroot=newroot --inherit-karg=root or something?

Or I guess the best place to bikeshed this would be https://github.com/containers/bootc/issues/404

[cgwalters]

Thanks for starting this! My desktop was having some issues and I wanted to do a factory reset, so I had some motivation to push this one farther. I ended up doing it as part of the install flow which I think feels cleaner.

New PR in https://github.com/bootc-dev/bootc/pull/1389