bootc-image-builder
bootc-image-builder copied to clipboard
osbuild
Switching between architectures (arm64 <-> amd64) requires a reboot: Error: failed to open 2048 locks in /libpod_lock: numerical result out of range
Trying to build an image for both amd64 and arm64. When I switch to the other architecture, irrespective of which I do first, this error occurs.
May be related to https://github.com/containers/podman/issues/9164, except I don't think the problem is in the host's /dev/shm/libpod_lock. Removing it does nothing , it just gets recreated. I need to reboot the system to "fix" this.
Here's what I'm doing:
sudo podman pull --platform=linux/${TARGET_ARCH} quay.io/centos-bootc/centos-bootc:stream9
sudo podman run --rm -it \
--platform=linux/${TARGET_ARCH} \
--privileged \
--pull=newer \
--security-opt label=type:unconfined_t \
-v $CONFIG_TOML:/config.toml:ro \
-v $OUTPUT:/output \
-v /var/lib/containers/storage:/var/lib/containers/storage \
quay.io/centos-bootc/bootc-image-builder:latest \
--rootfs xfs \
--target-arch ${TARGET_ARCH} \
--chown $OWNER \
--type qcow2 \
--use-librepo=True \
quay.io/centos-bootc/centos-bootc:stream9
(...)
[/] Manifest generation step
Message: Generating manifest manifest-qcow2.json
2025/05/12 11:26:05 error: cannot build manifest: failed to inspect the image: exit status 125, stderr:
Error: failed to open 2048 locks in /libpod_lock: numerical result out of range
bootc-image-builder no longer pulls images, make sure to pull it before running bootc-image-builder:
sudo podman pull quay.io/centos-bootc/centos-bootc:stream9
As a probably unrelated problem, cross-building for arm64 in my amd64 host is failing in the same stage but with a different error:
[-] Manifest generation step
Message: Generating manifest manifest-qcow2.json
2025/05/12 11:38:10 error: cannot build manifest: running quay.io/centos-bootc/centos-bootc:stream9 container failed: exit status 126
stderr:
Failed to re-execute libcrun via memory file descriptor
time="2025-05-12T11:38:10Z" level=error msg="Removing container 33ba1d09c2b9f590c75fd1dc71bc6f4197610fe18ded5053af6b23de664ea846 from runtime after creation failed"
Error: OCI runtime error: crun: Failed to re-execute libcrun via memory file descriptor
To summarize:
- Building for amd64 first succeeds, then for arm64 fails with the libpod_lock error.
- Building for arm64 first fails with the libcrun error, then building for amd64 fails with the libpod_lock error.
BTW this might be relevant:
❯ podman info
host:
arch: amd64
buildahVersion: 1.39.4
cgroupControllers:
- cpu
- memory
- pids
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: conmon-1:2.1.13-1
path: /usr/bin/conmon
version: 'conmon version 2.1.13, commit: 82de887596ed8ee6d9b2ee85e4f167f307bb569b'
cpuUtilization:
idlePercent: 98.57
systemPercent: 0.5
userPercent: 0.93
cpus: 16
databaseBackend: sqlite
distribution:
distribution: arch
version: unknown
eventLogger: journald
freeLocks: 2045
hostname: kuze
idMappings:
gidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 100000
size: 65536
uidmap:
- container_id: 0
host_id: 1000
size: 1
- container_id: 1
host_id: 100000
size: 65536
kernel: 6.14.6-arch1-1
linkmode: dynamic
logDriver: journald
memFree: 2201124864
memTotal: 29213159424
networkBackend: netavark
networkBackendInfo:
backend: netavark
dns:
package: aardvark-dns-1.14.0-1
path: /usr/lib/podman/aardvark-dns
version: aardvark-dns 1.14.0
package: netavark-1.14.1-1
path: /usr/lib/podman/netavark
version: netavark 1.14.1
ociRuntime:
name: crun
package: crun-1.21-1
path: /usr/bin/crun
version: |-
crun version 1.21
commit: 10269840aa07fb7e6b7e1acff6198692d8ff5c88
rundir: /run/user/1000/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
os: linux
pasta:
executable: /usr/bin/pasta
package: passt-2025_05_07.eea8a76-1
version: ""
remoteSocket:
exists: true
path: /run/user/1000/podman/podman.sock
rootlessNetworkCmd: pasta
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: /etc/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: false
slirp4netns:
executable: ""
package: ""
version: ""
swapFree: 34359734272
swapTotal: 34359734272
uptime: 5h 31m 9.00s (Approximately 0.21 days)
variant: ""
plugins:
authorization: null
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
- ipvlan
volume:
- local
registries:
<redacted>:
Blocked: false
Insecure: true
Location: <redacted>
MirrorByDigestOnly: false
Mirrors: null
Prefix: <redacted>
PullFromMirror: ""
search:
- docker.io
- quay.io
store:
configFile: /home/<redacted>/.config/containers/storage.conf
containerStore:
number: 2
paused: 0
running: 1
stopped: 1
graphDriverName: overlay
graphOptions: {}
graphRoot: /home/<redacted>/.local/share/containers/storage
graphRootAllocated: 1023117623296
graphRootUsed: 364196225024
graphStatus:
Backing Filesystem: btrfs
Native Overlay Diff: "true"
Supports d_type: "true"
Supports shifting: "false"
Supports volatile: "true"
Using metacopy: "false"
imageCopyTmpDir: /var/tmp
imageStore:
number: 100
runRoot: /run/user/1000/containers
transientStore: false
volumePath: /home/<redacted>/.local/share/containers/storage/volumes
version:
APIVersion: 5.4.2
Built: 1745099855
BuiltTime: Sat Apr 19 22:57:35 2025
GitCommit: be85287fcf4590961614ee37be65eeb315e5d9ff
GoVersion: go1.24.2
Os: linux
OsArch: linux/amd64
Version: 5.4.2
I'm facing same issue. In my case builds works, but the build fails with following output (x86 works ok):
sh-5.2# podman image list | grep 10.0
workshop-registry-quay-openshift-operators.apps.cluster-52sqt.52sqt.sandbox2533.opentlc.com/user99/myrhel 10.0-arm64 ced59704519d 58 minutes ago 1.5 GB
workshop-registry-quay-openshift-operators.apps.cluster-52sqt.52sqt.sandbox2533.opentlc.com/user99/myrhel 10.0 ced59704519d 58 minutes ago 1.5 GB
workshop-registry-quay-openshift-operators.apps.cluster-52sqt.52sqt.sandbox2533.opentlc.com/user99/myrhel 10.0-amd64 9aa6d322de86 About an hour ago 1.48 GB
sh-5.2# PLATFORM_ITEM=linux/arm64DIR_NAME=myrhel_10.0-anaconda-iso
FORMAT_ITEM=anaconda-iso
IMAGE=workshop-registry-quay-openshift-operators.apps.cluster-52sqt.52sqt.sandbox2533.opentlc.com/user99/myrhel:10.0
sudo podman run \
--platform=${PLATFORM_ITEM} \
--rm \
--privileged \
--pull=newer \
--security-opt label=type:unconfined_t \
-v ./config.toml:/config.toml:z \
-v /bootc-exports/$(echo "$PLATFORM_ITEM" | sed 's|linux/||')/$DIR_NAME:/output \
-v /var/lib/containers/storage:/var/lib/containers/storage \
quay.io/centos-bootc/bootc-image-builder:latest --target-arch $(echo "$PLATFORM_ITEM" | sed 's|linux/||') --type ${FORMAT_ITEM} --use-librepo=True $IMAGE
Generating manifest manifest-anaconda-iso.json
Manifest generation step
2025/06/16 11:28:43 error: cannot build manifest: failed to inspect the image: exit status 125, stderr:
Error: failed to open 2048 locks in /libpod_lock: numerical result out of range
bootc-image-builder no longer pulls images, make sure to pull it before running bootc-image-builder:
sudo podman pull workshop-registry-quay-openshift-operators.apps.cluster-52sqt.52sqt.sandbox2533.opentlc.com/user99/myrhel:10.0
Minimal reproducer:
sudo podman run \
--platform=linux/arm64 \
--privileged \
-v /var/lib/containers/storage:/var/lib/containers/storage \
--entrypoint bash \
quay.io/centos-bootc/bootc-image-builder:latest -c "mount -t devtmpfs devtmpfs /dev && podman ps"
Minimal potential fix?
sudo podman run \
--platform=linux/arm64 \
--privileged \
-v /var/lib/containers/storage:/var/lib/containers/storage \
--entrypoint bash \
quay.io/centos-bootc/bootc-image-builder:latest -c "mount -t devtmpfs devtmpfs /dev && mount -t tmpfs tmpfs /dev/shm && podman ps"
