lima icon indicating copy to clipboard operation
lima copied to clipboard

MacOS Rosetta broke on latest kernel Fedora 41 (rosetta error: unhandled auxillary vector type 29)

Open hasan4791 opened this issue 6 months ago • 20 comments

Though this is not lima issue, opening this issue for reference to users who are all facing similar issue.

After creating new instance,

[root@lima-chasan ~]# uname -a
Linux lima-chasan 6.11.4-301.fc41.aarch64 #1 SMP PREEMPT_DYNAMIC Sun Oct 20 15:21:05 UTC 2024 aarch64 GNU/Linux
[root@lima-chasan ~]#
[root@lima-chasan ~]# podman run -it --rm --platform linux/amd64 ubuntu bash
root@89049f16056a:/# ps -eaf
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  2 13:58 pts/0    00:00:00 /usr/bin/bash
root           4       1 50 13:58 pts/0    00:00:00 /usr/bin/ps -eaf
root@89049f16056a:/#

After updating the instance to latest, its not working

[root@lima-chasan ~]# uname -a
Linux lima-chasan 6.14.6-200.fc41.aarch64 #1 SMP PREEMPT_DYNAMIC Fri May  9 20:22:11 UTC 2025 aarch64 GNU/Linux
[root@lima-chasan ~]#
[root@lima-chasan ~]# podman run -it --rm --platform linux/amd64 ubuntu bash
rosetta error: unhandled auxillary vector type 29
 [root@lima-chasan ~]#

hasan4791 avatar May 27 '25 14:05 hasan4791

Please report to Apple

AkihiroSuda avatar May 28 '25 02:05 AkihiroSuda

@hasan4791 can you share the lima command to create the instance or the lima.yaml? I'm using rosetta with k8s template (ubuntu 24.04) without any issues.

nirs avatar Jun 06 '25 16:06 nirs

I'm using rosetta with k8s template (ubuntu 24.04) without any issues.

This issue is about Fedora 41 after installing latest updates. The Rosetta issue will be triggered by the Fedora kernel, and not by something in the Ubuntu container image (which doesn't include the kernel).

jandubois avatar Jun 06 '25 16:06 jandubois

@hasan4791 can you share the lima command to create the instance or the lima.yaml? I'm using rosetta with k8s template (ubuntu 24.04) without any issues.

Whats the kernel version btw? The issue is with the latest kernel.

hasan4791 avatar Jun 06 '25 16:06 hasan4791

Whats the kernel version btw?

Your commands above show a 6.14 kernel version.

I believe that Apple has fixed Rosetta for 6.11 and 6.12 kernels in macOS 15.5, but there are issues with 6.13 (and as you show here 6.14).

Can't remember where I saw the report about 6.13 errors though.

PS: Just realized that you were asking about @nirs' kernel version. 😄

jandubois avatar Jun 06 '25 16:06 jandubois

Can't remember where I saw the report about 6.13 errors though.

Found it: https://github.com/utmapp/UTM/discussions/6799#discussioncomment-13193067

jandubois avatar Jun 06 '25 16:06 jandubois

I suppose that we could use CentOS Stream for Podman, to get a less bleeding-edge kernel (but still a newer podman)

Depending on whether it is important to have the latest/greatest in a few days, as opposed to weeks or months (or years)

https://endoflife.date/podman

https://repology.org/project/podman/badges

But containerd (nerdctl) and docker are the latest versions, so I guess users are expecting that from template://podman ?

One way would be to use a "podman-full" similar to "nerdctl-full", but that would need a maintainer to take on the burden

afbjorklund avatar Jun 06 '25 17:06 afbjorklund

podman is temporarily switching back to qemu-user-static because of this bug: https://blog.podman.io/2025/06/podman-and-apple-rosetta/ This bug might be fixed in macOS 26 beta

cfergeau avatar Jun 10 '25 09:06 cfergeau

Detailed steps to reproduce:

$ limactl start --rosetta template://fedora-41

$ limactl shell fedora-41
[vm] $ uname -a
[vm] $ podman run -it --rm --platform linux/amd64 ubuntu bash

# exit the container’s shell

[vm] $ sudo dnf -y update
[vm] $ sudo systemctl reboot

$ limactl shell fedora-41
[vm] $ uname -a
[vm] $ podman run -it --rm --platform linux/amd64 ubuntu bash

These steps worked on my macOS 26 beta install.

cfergeau avatar Jun 17 '25 10:06 cfergeau

These steps worked on my macOS 26 beta install.

@cfergeau Do you mean this is fixed in macOS 26 beta?

nirs avatar Jun 17 '25 14:06 nirs

These steps worked on my macOS 26 beta install.

@cfergeau Do you mean this is fixed in macOS 26 beta?

Yes, this appears to be fixed, the container image starts correctly with the newer kernel. I no longer have a macOS 15 install so I can’t double check these exact steps fail with the unhandled auxiliary vector error.

cfergeau avatar Jun 17 '25 14:06 cfergeau

Thank you @cfergeau for keeping us updated.

hasan4791 avatar Jun 17 '25 14:06 hasan4791

I no longer have a macOS 15 install so I can’t double check these exact steps fail with the unhandled auxiliary vector error.

They do:

[jan@lima-fedora-41 lima]$ uname -a
Linux lima-fedora-41 6.14.11-200.fc41.aarch64 #1 SMP PREEMPT_DYNAMIC Tue Jun 10 16:46:42 UTC 2025 aarch64 GNU/Linux
[jan@lima-fedora-41 lima]$ podman run -it --rm --platform linux/amd64 ubuntu bash
rosetta error: unhandled auxillary vector type 29
 [jan@lima-fedora-41 lima]$

Latest macOS 15:

❯ sw_vers
ProductName:		macOS
ProductVersion:		15.5
BuildVersion:		24F74

So good to know it is fixed in 26. Hopefully also in 15.6, but I don't have the beta installed.

jandubois avatar Jun 17 '25 16:06 jandubois

Hopefully also in 15.6, but I don't have the beta installed.

Sadly it is not fixed in the latest 15.6 beta:

% sw_vers
ProductName:		macOS
ProductVersion:		15.6
BuildVersion:		24G5054d

jandubois avatar Jun 17 '25 17:06 jandubois

As mentioned by @jandubois its working with Macos 26 version and not sure about 15.7 version.

Apart from this, if there is a way to control this based on the mac version, in future we can have a fallback mechanism or we need to assess to move to centos instead of fedora. Open for suggestions here!!

hasan4791 avatar Sep 17 '25 04:09 hasan4791

in future we can have a fallback mechanism or we need to assess to move to centos instead of fedora

we have images for both centos-stream and almalinux, in addition to fedora. So that's already available

afbjorklund avatar Sep 17 '25 05:09 afbjorklund

I tried to replace .base[0], but it is not supported:

$ limactl create template://podman --set '.base[0]="template://_images/centos-stream-9"' --tty=false
INFO[0000] Terminal is not available, proceeding without opening an editor 
WARN[0000] vmType vz: ignoring [Base]                   
FATA[0000] field `base` must be empty for YAML validation 

norio-nomura avatar Sep 17 '25 05:09 norio-nomura

not sure about 15.7 version

It has not been fixed, and I would expect that it will never be fixed for macOS 15.

❯ sw_vers
ProductName:		macOS
ProductVersion:		15.7
BuildVersion:		24G222

❯ l shell fedora-41
[jan@lima-fedora-41 lima]$ uname -a
Linux lima-fedora-41 6.16.7-100.fc41.aarch64 #1 SMP PREEMPT_DYNAMIC Thu Sep 11 17:21:49 UTC 2025 aarch64 GNU/Linux
[jan@lima-fedora-41 lima]$ podman run -it --rm --platform linux/amd64 ubuntu bash
Resolved "ubuntu" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/ubuntu:latest...
Getting image source signatures
Copying blob 953cdd413371 done   |
Copying config 6d79abd4c9 done   |
Writing manifest to image destination
rosetta error: unhandled auxillary vector type 29

jandubois avatar Sep 17 '25 07:09 jandubois

I tried to replace .base[0], but it is not supported:

That can't work; the base templates are merged as part of the template loading, before the YQ expressions from the CLI are evaluated.

It is easy enough to make a copy of the template and edit the base.

jandubois avatar Sep 17 '25 07:09 jandubois

It is easy enough to make a copy of the template and edit the base.

l tmpl copy template://podman - | l yq '.base[0]="template://_images/centos-stream-9"' | l start --name podman -

jandubois avatar Sep 17 '25 07:09 jandubois