bootc/rpm-ostree update fails on Importing: Unencapsulating base: failed to invoke method FinishPipe with zstd:chunked image

[travier]

Updating to a zstd:chunk'ed image using bootc/rpm-ostree fails with:

$ sudo bootc update
layers already present: 42; layers needed: 24 (1.3 GB)
Fetched layers: 0 B in 2 seconds (0 B/s)
ERROR Upgrading: Pulling: Importing: Unencapsulating base: failed to invoke method FinishPipe: failed to invoke method FinishPipe: write |1: broken pipe

$ sudo bootc status
No staged image present
Current booted image: quay.io/travier/fedora-kinoite:latest
    Image version: 41.20241207.0 (2024-12-07 04:10:35.798585027 UTC)
    Image digest: sha256:40efa2aa91e2ad4c3b75218cce8463d78193f2b35b2dfc972d412fba5eaaccb7
Current rollback image: quay.io/travier/fedora-kinoite:latest
    Image version: 41.20241202.0 (2024-12-02 04:11:16.290882830 UTC)
    Image digest: sha256:1807aefcb830c8952c1ecebaaab28900ef48ea5f8075e4a6b34a38d788f1da7e

$ sudo rpm-ostree update
Pulling manifest: ostree-image-signed:docker://quay.io/travier/fedora-kinoite:latest
Importing: ostree-image-signed:docker://quay.io/travier/fedora-kinoite:latest (digest: sha256:1f27336996fc663ad0ef3f448eef21edfa71322931fde0fe9f577068736aa2d5)
ostree chunk layers already present: 43
ostree chunk layers needed: 22 (1.2 GB)
custom layers needed: 1 (38.8 MB)
[0/23] Fetching ostree chunk f3c0f64a5cdf6196496 (50.4 MB)... done
error: Importing: Unencapsulating base: failed to invoke method FinishPipe: failed to invoke method FinishPipe: write |1: broken pipe

I had been using zstd:chunked images with success (https://github.com/travier/fedora-kinoite) until rpm-ostree 2024.9-1.fc41 landed in F41 recently. Turning the images compression back to zstd worked as a workaround which suggests that this is a zstd:chunk specific issue. This is very similar to https://github.com/containers/bootc/issues/509 but only happened recently.

This also impacts bootc-1.1.2-2.fc41.x86_64 which suggests that this is a regression. It used to work with bootc but I don't remember the version it worked with.

• Last know good:
  • rpm-ostree 2024.8-1.fc41
• Know bad:
  • rpm-ostree 2024.9-1.fc41
  • bootc-1.1.2-2.fc41.x86_64

I haven't been able to pinpoint/bissect exactly which commit introduced this regression yet. Reporting this so that it does not get lost and I'll try to bissect the regression later.

[cgwalters]

This also impacts bootc-1.1.2-2.fc41.x86_64 which suggests that this is a regression

It may somehow be a regression rom https://github.com/ostreedev/ostree-rs-ext/pull/677

[pmtk]

Hey, I believe we started observing this in our MicroShift CI for centos bootc images. Here's the issue: https://issues.redhat.com/browse/USHIFT-5546

[jeckersb]

This should be fixed once https://github.com/bootc-dev/bootc/pull/1247 merges

[OmegaSquad82]

I confirm that, with the version of bootc (and rpm-ostree) shipped with bazzite, it works to switch to a zstd:chunked image.

$ sudo bootc status
● Booted image: ghcr.io/omegasquad82/buttgenbachit:latest-zstchd
        Digest: sha256:522715efa9883af599cac599eba9f15db97e4c13059e1e4ff5a3f2c17f27baaf (amd64)
       Version: 42.250602 (2025-06-02T06:33:45Z)

  Rollback image: ghcr.io/omegasquad82/buttgenbachit:latest
          Digest: sha256:a08b971a0283cbbfcbef3602659e310e09de5122cd0b1178d1d5ce3bc2f44206 (amd64)
         Version: 42.250530 (2025-05-30T06:31:46Z)

$ rpm-ostree status
State: idle
Deployments:
● ostree-image-signed:docker://ghcr.io/omegasquad82/buttgenbachit:latest-zstchd
                   Digest: sha256:522715efa9883af599cac599eba9f15db97e4c13059e1e4ff5a3f2c17f27baaf
                  Version: 42.250602 (2025-06-02T06:33:45Z)

  ostree-image-signed:docker://ghcr.io/omegasquad82/buttgenbachit:latest
                   Digest: sha256:a08b971a0283cbbfcbef3602659e310e09de5122cd0b1178d1d5ce3bc2f44206
                  Version: 42.250530 (2025-05-30T06:31:46Z)

[cgwalters]

Thanks, closing!