Podman-machine default installer, won't resolve/progress on a download issue (lost connection, incomplete file...)

[Peyman-Borhani]

Bug description

When initializing podman-desktop, the installer is building a default-machine.

A file download process took long (maybe a connection issue), so I exit it. On the next retry, it results in: error: exit code 125

In terminal executing podman machine init resulted in unable to extract and unexpected EOF error.

This lead me to find-delete downloaded file, which solved the issue, and installer completed on the next try.

My assumption is podman or podman-desktop app not making a download completion/verification record, and therefore not deleting/overwriting it on next attempts, instead it tries to extract the incomplete file.

Is best if the file is checked and verified, before extraction, and if not retry overwrite invalid file.

Thanks.

Operating system

Windows 10 x64 (ver 22H2 / Build 19045.4412)

Installation Method

Installer from website/GitHub releases

Version

1.10.2

Steps to reproduce

Break the download in default machine install process. (or corrupt the file)

Relevant log output

$ podman machine init
time="2024-05-28T10:11:56+03:30" level=error msg="Error extracting compressed file: \"unexpected EOF\""
Error: unexpected EOF

### Additional context

_No response_

[jeffmaury]

I think it's an upstream issue (Podman)

[baude]

please submit the results of podman info; from this, i cannot tell if you are using hyperv or wsl and that is a relevant detail.

[baude]

also, when you said you exit, is this a control-c

[Peyman-Borhani]

please submit the results of podman info; from this, i cannot tell if you are using hyperv or wsl and that is a relevant detail.

@baude WSL is used

podman info log::

$ podman info host: arch: amd64 buildahVersion: 1.35.4 cgroupControllers:

• cpuset
• cpu
• cpuacct
• blkio
• memory
• devices
• freezer
• net_cls
• perf_event
• net_prio
• hugetlb
• pids
• rdma
• misc cgroupManager: cgroupfs cgroupVersion: v1 conmon: package: conmon-2.1.10-1.fc40.x86_64 path: /usr/bin/conmon version: 'conmon version 2.1.10, commit: ' cpuUtilization: idlePercent: 99.78 systemPercent: 0.13 userPercent: 0.09 cpus: 12 databaseBackend: sqlite distribution: distribution: fedora variant: container version: "40" eventLogger: journald freeLocks: 2048 hostname: ___ idMappings: gidmap: null uidmap: null kernel: 5.15.146.1-microsoft-standard-WSL2 linkmode: dynamic logDriver: journald memFree: 7281381376 memTotal: 8247300096 networkBackend: netavark networkBackendInfo: backend: netavark dns: package: aardvark-dns-1.10.0-1.fc40.x86_64 path: /usr/libexec/podman/aardvark-dns version: aardvark-dns 1.10.0 package: netavark-1.10.3-3.fc40.x86_64 path: /usr/libexec/podman/netavark version: netavark 1.10.3 ociRuntime: name: crun package: crun-1.15-1.fc40.x86_64 path: /usr/bin/crun version: |- crun version 1.15 commit: e6eacaf4034e84185fd8780ac9262bbf57082278 rundir: /run/crun spec: 1.0.0 +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL os: linux pasta: executable: /usr/bin/pasta package: passt-0^20240510.g7288448-1.fc40.x86_64 version: | pasta 0^20240510.g7288448-1.fc40.x86_64 Copyright Red Hat GNU General Public License, version 2 or later https://www.gnu.org/licenses/old-licenses/gpl-2.0.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. remoteSocket: exists: true path: /run/podman/podman.sock security: apparmorEnabled: false capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT rootless: false seccompEnabled: true seccompProfilePath: /usr/share/containers/seccomp.json selinuxEnabled: false serviceIsRemote: true slirp4netns: executable: "" package: "" version: "" swapFree: 2147483648 swapTotal: 2147483648 uptime: 4h 41m 32.00s (Approximately 0.17 days) variant: "" plugins: authorization: null log:
  • k8s-file
  • none
  • passthrough
  • journald network:
  • bridge
  • macvlan
  • ipvlan volume:
  • local registries: search:
  • docker.io store: configFile: /usr/share/containers/storage.conf containerStore: number: 0 paused: 0 running: 0 stopped: 0 graphDriverName: overlay graphOptions: overlay.imagestore: /usr/lib/containers/storage overlay.mountopt: nodev,metacopy=on graphRoot: /var/lib/containers/storage graphRootAllocated: 1081101176832 graphRootUsed: 848224256 graphStatus: Backing Filesystem: extfs Native Overlay Diff: "false" Supports d_type: "true" Supports shifting: "false" Supports volatile: "true" Using metacopy: "true" imageCopyTmpDir: /var/tmp imageStore: number: 0 runRoot: /run/containers/storage transientStore: false volumePath: /var/lib/containers/storage/volumes version: APIVersion: 5.0.3 Built: 1715299200 BuiltTime: Fri May 10 03:30:00 2024 GitCommit: "" GoVersion: go1.22.2 Os: linux OsArch: linux/amd64 Version: 5.0.3

Exit by ctrl-c ? no, was in Podman-desktop app: 1st time canceled, retry... next end task, retry... then restart system, retry. had same issue.

Further I searched the podman folders, couldn't find a log on the event, but tracked the downloading file: wslv20240520154143-5.0-rootfs-amd64.tar.zst deleting it fixed the problem, setup completes and works well.

My reason (assumption): podman desktop in-app install of machine-default stuck on extraction... in CLI: podman machine init gave unexpected EOF error. I locate the file and noticed its size was incorrect (incomplete/damaged...) then as podman-cli/desktop-app didn't remove or overwrite it on retry, I delete the file, and issue fixed, machine setup completed in desktop app, and the downloaded file size was correct.

So I thought it might be useful to share my findings, and suggest adding a function to check the downloaded file, to verify completion, file size, checksum...etc before it proceeds to extraction.

[baude]

Ill assign myself this one though probably wont be until next week (bug week) that I get to it. We can do a digest verification but I am concerned about the cost and how often we do it (i.e. reaching out to the registry (each time would be two different requests) and then actually checksum'ing the local file. Perhaps it can be done on download only and if that does not match, we throw away the downloaded file and error out?

the current approach does show a weakness in how we determine the local cached file though. in the sense that if, as is pointed out by the person who wrote this issue, the downloaded file is corrupt, we get stuck in this loop of sorts. podman machine reset would get you out of that, but many people dont know about that command or use it yet.

[github-actions[bot]]

A friendly reminder that this issue had no activity for 30 days.

[pzmarzly]

I just ran into this issue when installing Podman Desktop for the first time.

> podman machine init

Extracting compressed file: podman-machine-default-amd64: done
Error: unexpected EOF

Running podman machine reset && podman machine init indeed seems to have helped. The image was v20240717061616-5.1-rootfs-amd64.tar.zst, podman version podman.exe version 5.1.1

[raatti]

Can confirm with clean installation of podman desktop using Winget:

PS C:\Users\jani> winget install -e --id RedHat.Podman-Desktop
Found Podman Desktop [RedHat.Podman-Desktop] Version 1.11.1
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://github.com/containers/podman-desktop/releases/download/v1.11.1/podman-desktop-1.11.1-setup-x64.exe
  ██████████████████████████████   145 MB /  145 MB
Successfully verified installer hash
Starting package install...
Installer failed with exit code: 3221225477

Error is the same: Error: unexpected EOF

Resetting it manually from PS fixes situation:

PS C:\Program Files\RedHat\Podman> .\podman.exe machine reset
Warning: this command will delete all existing Podman machines
and all of the configuration and data directories for Podman machines

The following machine(s) will be deleted:


Are you sure you want to continue? [y/N] y
PS C:\Program Files\RedHat\Podman> .\podman.exe machine init
Downloading VM image: v20240728061533-5.1-rootfs-amd64.tar.zst: done
Extracting compressed file: podman-machine-default-amd64: done
Importing operating system into WSL (this may take a few minutes on a new WSL install)...
Import in progress, this may take a few minutes.
The operation completed successfully.
Configuring system...
Machine init complete
To start your machine run:

        podman machine start

[inknos]

Currently, the issue is caused by machines being pulled from a tar.zst disk rather than OCI images.

This change will fix this issue. I just tested it now and I am working on some tests for it

[oddwatcher]

This BUG is still persistent on podman.exe version 5.2.4 windows WSL if the downloading was interrupted the init process won't resume automatically.

[inknos]

Correct. This change is going to land in podman 5.3. The micro release 5.2.4 targets smaller changes/fixes and this is too big to land in 5.2.x

[TurnOffNOD]

Correct. This change is going to land in podman 5.3. The micro release 5.2.4 targets smaller changes/fixes and this is too big to land in 5.2.x

When will podman 5.3 release?

[fmancardi]

My 5 cents working on Windows I've found where podman stores the containers

c:\Users<YOUR-USER>.local\share\containers\

That contains the following folders podman podman-desktop storage

then I've deleted the folder podman.
Connected the PC to the network using cable.
Rerun podman machine init

Got these messages

Downloading VM image: v20241213061153-5.2-rootfs-amd64.tar.zst: done
Extracting compressed file: podman-machine-default-amd64: done
Importing operating system into WSL (this may take a few minutes on a new WSL install)...
Import in progress, this may take a few minutes.
The operation completed successfully.
Configuring system...
Machine init complete
To start your machine run:

        podman machine start

Hope this can help

[wbunthof]

I've got the same issue. However, by deleting the c:\Users<YOUR-USER>.local\share\containers\podman folder it was resolved for me as well.

[hritikshinde13]

I've got the same issue. However, by deleting the c:\Users<YOUR-USER>.local\share\containers\podman folder it was resolved for me as well.

Helped me thanks