Cannot start custom image toolbox after using flatcar-reset

[ryester19]

I have the following contents present in my Butane/Ignition config:

storage:
  files:
    - path: /home/core/.toolboxrc
      mode: 0644
      contents:
        inline: |
          TOOLBOX_DOCKER_IMAGE=docker.io/library/alpine
          TOOLBOX_USER=root

I have no issues launching the toolbox right after booting Flatcar for the first time on a new installation.

When I use flatcar-reset (without arguments), and then try to launch the toolbox after reboot, I'm met with this error:

unpacking linux/amd64 sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c...
ctr: failed to prepare extraction snapshot "extract-303398294-eHgJ sha256:08000c18d16dadf9553d747a58cf44023423a9ab010aab96cf263d2216b8b350": failed to rename: rename /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4126173960 /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1: file exists: unknown

Removing the contents of /var/lib/containerd manually does allow me to start the Alpine toolbox again

[tormath1]

Hi @ryester19 and thanks for the report,

When you restart after a flatcar-reset it's similar to reboot from a fresh install so your custom configuration is wiped out.

storage:
  files:
    - path: /home/core/.toolboxrc
      mode: 0644
      contents:
        inline: |
          TOOLBOX_DOCKER_IMAGE=docker.io/library/alpine
          TOOLBOX_USER=root

I think before the error message:

ctr: failed to prepare extraction snapshot "extract-303398294-eHgJ sha256:08000c18d16dadf9553d747a58cf44023423a9ab010aab96cf263d2216b8b350": failed to rename: rename /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4126173960 /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1: file exists: unknown

You might even see that it does not use the alpine image but the default fedora one.

If you want to flatcar-reset without running Ignition again, you can try to create your configuration here: /etc/default/toolbox then run flatcar-reset -K /etc/default/toolbox.

Let us know how things are working.

[ryester19]

When you restart after a flatcar-reset it's similar to reboot from a fresh install so your custom configuration is wiped out

That doesn't appear to be the case for me. The .toolboxrc is still present after a restart, even after running flatcar-reset w/o arguments. Here's the full command output.

core@localhost ~ $ cat .toolboxrc
TOOLBOX_DOCKER_IMAGE=docker.io/library/alpine
TOOLBOX_USER=root
core@localhost ~ $ toolbox
docker.io/library/alpine:latest:                                                  resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:1c4eef651f65e2f7daee7ee785882ac164b02b78fb74503052a26dc061c90474: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:aded1e1a5b3705116fa0a92ba074a5e0b0031647d9c315983ccba2ee5428ec8b:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:f18232174bc91741fdf3da96d85011092101a032a93a388b79e99e69c2d5c870:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.8 s                                                                    total:  10.0 K (12.5 KiB/s)           
unpacking linux/amd64 sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c...
ctr: failed to prepare extraction snapshot "extract-468637500-qwx8 sha256:08000c18d16dadf9553d747a58cf44023423a9ab010aab96cf263d2216b8b350": failed to rename: rename /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/new-4122956533 /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/1: file exists: unknown

It looks to me like the ignition config got "baked in" when I ran flatcar-install -d <device> -i <ignition_file>. Is that suppose to happen?

[tormath1]

Oh, you used flatcar-install. Can you describe the full workflow then?