podman machine start hangs on macOS (x86_64)

[crahan]

Issue Description

Starting the podman machine VM on macOS Intel Sequoia hangs right after the bootloader screen with the message Booting 'Fedora CoreOS 41.20250105.3.0 (ostree:0). I tried the same on an Apple Silicon device and the VM boots fine, showing EFI statements, then going through its normal boot process.

I tried the Podman version in Homebrew, the offical CLI installer, and installing through Podman Desktop. In all cases it halted immediately after the bootloader screen.

Below are screenshots of the podman machine start --log-level debug output and the VM itself showing where it hangs. Console debug output does not contain any errors.

Steps to reproduce the issue

Steps to reproduce the issue

1. Run podman machine init.
2. Run podman machine start --log-level debug.
3. VM boot process halts right after the bootloader screen.

Describe the results you received

The Podman Machine VM bootup process hangs.

Describe the results you expected

The Podman Machine VM boots.

podman info output

No `podman info` output due to no VM.
Tested with Podman versions 5.3.2 and 5.3.1 with similar results.

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

macOS Sequoia on a Macbook Pro Intel device.

Additional information

No response

[szymon3]

Same issue here, same OS and hardware

$ podman info
OS: darwin/amd64
provider: applehv
version: 5.3.2

Cannot connect to Podman. Please verify your connection to the Linux system using `podman system connection list`, or try `podman machine init` and `podman machine start` to manage a new Linux VM
Error: unable to connect to Podman socket: Get "http://d/v5.3.2/libpod/_ping": dial unix /var/folders/wt/fy6rz4y53l5dz9b0s4w5wnv80000gn/T/storage-run-501/podman/podman.sock: connect: no such file or directory

EDIT: I did some testing and the latest working release for me was 5.2.5.

[lielfr]

I have the issue on an Intel Mac as well. It seems to work ok on an ARM mac though

[baude]

thank you for the reports, we are trying to find someone with an intel mac and see if we can reproduce and fix. otherwise, i may be asking lots of questions here ;)

[benoitf]

I'm able to start a podman machine on macOS/Intel but using 14.6.1 (23G93) version (Sonoma)

will try to upgrade the OS

[benoitf]

uname -a
Darwin fbenoit-mac 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:00 PDT 2024; root:xnu-10063.141.2~1/RELEASE_X86_64 x86_64

$ podman version 
Client:       Podman Engine
Version:      5.3.2
API Version:  5.3.2
Go Version:   go1.23.5
Git Commit:   85043bb1a3818102194afa82845cb63841067c9c
Built:        Wed Jan 22 14:40:10 2025
OS/Arch:      darwin/amd64

Server:       Podman Engine
Version:      5.3.2
API Version:  5.3.2
Go Version:   go1.23.4
Built:        Wed Jan 22 01:00:00 2025
OS/Arch:      linux/amd64

[k-paz]

you might also check your binaries

file /opt/podman/bin/krunkit - gives me WRONG/arm version: /opt/podman/bin/krunkit: Mach-O 64-bit executable arm64

while I think it also should be the x86_64/AMD64 version, so I've just opened a separate case for that issue: https://github.com/containers/podman/issues/25145

if you agree then please support the case resolution.

[benoitf]

upgraded the system to latest macOS (Sequoia 15.3)

podman machine init
Looking up Podman Machine image at quay.io/podman/machine-os:5.3 to create VM
Extracting compressed file: podman-machine-default-amd64.raw: done
Machine init complete
To start your machine run:

	podman machine start

 ~  podman machine start
Starting machine "podman-machine-default"

This machine is currently configured in rootless mode. If your containers
require root permissions (e.g. ports < 1024), or if you run into compatibility
issues with non-podman clients, you can switch using the following command:

	podman machine set --rootful

API forwarding listening on: /var/run/docker.sock
Docker API clients default to this address. You do not need to set DOCKER_HOST.

Machine "podman-machine-default" started successfully
 ~  podman run quay.io/podman/hello 
Trying to pull quay.io/podman/hello:latest...
Getting image source signatures
Copying blob sha256:81df7ff16254ed9756e27c8de9ceb02a9568228fccadbf080f41cc5eb5118a44
Copying config sha256:5dd467fce50b56951185da365b5feee75409968cbab5767b9b59e325fb2ecbc0
Writing manifest to image destination
!... Hello Podman World ...!

         .--"--.
       / -     - \
      / (O)   (O) \
   ~~~| -=(,Y,)=- |
    .---. /`  \   |~~
 ~/  o  o \~~~~.----. ~~
  | =(X)= |~  / (O (O) \
   ~~~~~~~  ~| =(Y_)=-  |
  ~~~~    ~~~|   U      |~~

Project:   https://github.com/containers/podman
Website:   https://podman.io
Desktop:   https://podman-desktop.io
Documents: https://docs.podman.io
YouTube:   https://youtube.com/@Podman
X/Twitter: @Podman_io
Mastodon:  @[email protected]

[benoitf]

I'm using vfkit there, are you using krunkit ?

[benoitf]

$ podman machine ls
NAME                     VM TYPE     CREATED        LAST UP            CPUS        MEMORY      DISK SIZE
podman-machine-default*  applehv     3 minutes ago  Currently running  6           2GiB        100GiB
```

[crahan]

I'm using vfkit there, are you using krunkit ?

Happy to test. How can I verify or force vfkit vs krunkit?

[crahan]

Output from my screenshots above seem to indicate I'm using vfkit.

[benoitf]

CONTAINERS_MACHINE_PROVIDER=applehv podman machine init foo
CONTAINERS_MACHINE_PROVIDER=applehv podman machine start foo

[benoitf]

using libkrun I have expected

CONTAINERS_MACHINE_PROVIDER=libkrun podman machine start kr
Starting machine "kr"
ERRO[0000] process 2253 has not ended
Error: fork/exec /opt/podman/bin/krunkit: bad CPU type in executable

[benoitf]

can you try podman machine reset before trying again init/start

[crahan]

Here's the output

❯ podman --version
podman version 5.3.2

~
❯ which podman
/opt/podman/bin/podman

~
❯ CONTAINERS_MACHINE_PROVIDER=vfkit podman machine init foo
Error: unknown VMType `vfkit`

[benoitf]

yes sorry, it's applehv not vfkit

[benoitf]

but try podman machine reset, first

[crahan]

UPDATE: this is the output after running podman machine reset.

❯ CONTAINERS_MACHINE_PROVIDER=applehv podman machine init foo
Looking up Podman Machine image at quay.io/podman/machine-os:5.3 to create VM
Getting image source signatures
Copying blob c81785cd451b done   |
Copying config 44136fa355 done   |
Writing manifest to image destination
c81785cd451baeab0f16dce777a70f050caa74f8868bb80e74df10940b0093f6
Extracting compressed file: foo-amd64.raw: done
Machine init complete
To start your machine run:

        podman machine start foo


~ took 30s
❯ podman machine list
NAME        VM TYPE     CREATED         LAST UP     CPUS        MEMORY      DISK SIZE
foo*        applehv     42 seconds ago  Never       4           2GiB        100GiB

~
❯ CONTAINERS_MACHINE_PROVIDER=applehv podman machine start foo --log-level debug
INFO[0000] podman filtering at log level debug
DEBU[0000] Using Podman machine with `applehv` virtualization provider
...
INFO[0000] Exposing vsock port 1025 on /var/folders/lh/z0pqwz5918z836csbnx5w32w0000gn/T/podman/foo.sock (listening)
INFO[0000] Exposing vsock port 1024 on /Users/crahan/.local/share/containers/podman/machine/applehv/foo-ignition.sock (listening)
INFO[0000] waiting for VM to stop
2025-01-28 19:17:32.798 vfkit[24977:2192211] +[IMKClient subclass]: chose IMKClient_Modern
2025-01-28 19:17:32.798 vfkit[24977:2192211] +[IMKInputSession subclass]: chose IMKInputSession_Modern

[benoitf]

https://github.com/user-attachments/assets/4721874d-7e38-4811-ae68-29d6844e79a2

[crahan]

In my case the VM window doesn't go past the Booting... message. Even now, several minutes later, it's still at that same spot.

When I try on Apple Silicon instead, it works as expected.

[speedbunny]

Same issue here. Not really a podman user. Only commands I use are podman machine init and podman machine start.

podman machine start now hangs whereas it worked just fine and dandy for the last three months.

Tried fresh install / uninstall via Homebrew and also installed from the amd64 dmg after that. Downloaded desktop to try and troubleshoot.

podman --version podman version 5.3.2

podman info OS: darwin/amd64 provider: applehv version: 5.3.2

which podman /opt/podman/bin/podman

CONTAINERS_MACHINE_PROVIDER=applehv podman machine init foo podman machine start foo also hung (as suggested above)

As a basic user, I just want to be able to spin up a pod and network. Having to use Docker for now.

[baude]

i can also report it worked here ... let's take homebrew out of the equation and only use the released installers? i downloaded the amd64 installer. Are folks that are failing using the universal installer?

[benoitf]

for the record, I tried using a macbook pro 2018

[crahan]

@baude I can confirm my testing and ouput are from the official Podman CLI installer.

2020 Macbook Pro here (although I just noticed I'm on Sequia 15.2).

[baude]

with everyone being so active, would you be willing to join a gmeet to see if we can get to the bottom of this?

[crahan]

with everyone being so active, would you be willing to join a gmeet to see if we can get to the bottom of this?

Yeah, that would make sense!

[baude]

meet.google.com/aga-rrwq-qrn

[speedbunny]

with everyone being so active, would you be willing to join a gmeet to see if we can get to the bottom of this?

sure, not as if I'll be much help - currently updating to 15.3 to see if that helps.

[crahan]

Update to 15.3 finished, but unfortunately the issue remains for me.

[baude]

@crahan can you humor us and remove the current installation, install amd64 only binaries, podman machine reset, and try again ?