amazonlinux
[Bug] - Kernel 6.12 will not boot in VMWare Workstation
Tried upgrading the kernel to 6.12 and the VM will not boot inside VMWare Workstation
To Reproduce
- Load latest image into VMWare Workstation
- Install and set kernel 6.12 as default using instructions provided in release notes
- Reboot and attempt loading kernel 6.12
Expected behavior
- System would boot to login prompt
Desktop (please complete the following information):
- OS: Windows 11 Host
- VMWare Workstation 17.6.3 (latest version)
Additional context
- Gets stuck at the 'Booting the kernel (entry_offset..."
- After about 2 minutes it starts printing a loop to screen messages containing "dracut-initqueue: starting timeout scripts" and "systemd-cryptsetup"
- Auto powers off after about 5 minutes
@ccassidy-vaisala after installing kernel6.12-6.12.20-23.97.amzn2023.x86_64 the vm rebooted into the new kernel w/out issues:
[ 0.000000] Linux version 6.12.20-23.97.amzn2023.x86_64 (mockbuild@ip-10-0-61-160) (gcc (GCC) 11.5.0 20240719 (Red Hat 11.5.0-5), GNU ld version 2.41-50.amzn2023.0.2) #1 SMP PREEMPT_DYNAMIC Tue Mar 25 09:14:43 UTC 2025
[ 0.000000] Command line: BOOT_IMAGE=(hd0,gpt1)/boot/vmlinuz-6.12.20-23.97.amzn2023.x86_64 root=UUID=df52c2d8-b7f9-45c9-8c98-d0e1867d3820 ro console=ttyS0,115200n8 console=tty0 rd.emergency=poweroff rd.shell=0 selinux=1 security=selinux quiet
vm info:
[ec2-user@cernobyl ~]$ hostnamectl
Static hostname: cernobyl
Icon name: computer-vm
Chassis: vm 🖴
Machine ID: 7cfd750942ee400f9270b70e77c97c8e
Boot ID: 425a5e1f3117448da44ad4aeb3d24983
Virtualization: vmware
Operating System: Amazon Linux 2023.7.20250331
CPE OS Name: cpe:2.3:o:amazon:amazon_linux:2023
Kernel: Linux 6.12.20-23.97.amzn2023.x86_64
Architecture: x86-64
Hardware Vendor: VMware, Inc.
Hardware Model: VMware Virtual Platform
Firmware Version: 6.00
Host is Win11 + VMware Workstation 17.6.3 build-24583834
Kernel installation log:
[ec2-user@cernobyl ~]$ sudo dnf install kernel6.12 kernel6.12-modules-extra
Last metadata expiration check: 0:03:21 ago on Thu Apr 3 19:00:47 2025.
Dependencies resolved.
========================================================================================================================
Package Architecture Version Repository Size
========================================================================================================================
Installing:
kernel6.12 x86_64 6.12.20-23.97.amzn2023 amazonlinux 35 M
kernel6.12-modules-extra x86_64 6.12.20-23.97.amzn2023 amazonlinux 4.3 M
Transaction Summary
========================================================================================================================
Install 2 Packages
Total download size: 39 M
Installed size: 191 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): kernel6.12-modules-extra-6.12.20-23.97.amzn2023.x86_64.rpm 7.2 MB/s | 4.3 MB 00:00
(2/2): kernel6.12-6.12.20-23.97.amzn2023.x86_64.rpm 25 MB/s | 35 MB 00:01
------------------------------------------------------------------------------------------------------------------------
Total 26 MB/s | 39 MB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : kernel6.12-6.12.20-23.97.amzn2023.x86_64 1/2
Running scriptlet: kernel6.12-6.12.20-23.97.amzn2023.x86_64 1/2
Installing : kernel6.12-modules-extra-6.12.20-23.97.amzn2023.x86_64 2/2
Running scriptlet: kernel6.12-modules-extra-6.12.20-23.97.amzn2023.x86_64 2/2
Running scriptlet: kernel6.12-6.12.20-23.97.amzn2023.x86_64 2/2
dracut[E]: Module 'systemd-pcrphase' depends on 'tpm2-tss', which can't be installed
Checking Secure Boot revocations...
efivar: show variable: Function not implemented
Not a UEFI instance
Running scriptlet: kernel6.12-modules-extra-6.12.20-23.97.amzn2023.x86_64 2/2
Verifying : kernel6.12-6.12.20-23.97.amzn2023.x86_64 1/2
Verifying : kernel6.12-modules-extra-6.12.20-23.97.amzn2023.x86_64 2/2
Installed:
kernel6.12-6.12.20-23.97.amzn2023.x86_64 kernel6.12-modules-extra-6.12.20-23.97.amzn2023.x86_64
Complete!
To stop at the boot screen, start your vm with a working kernel and edit /boot/grub2/grub.cfg by changing the timeout setting. Look around line 75:
75 if [ x$feature_timeout_style = xy ] ; then
76 set timeout_style=menu
77 set timeout=5
78 # Fallback normal timeout code in case the timeout_style feature is
79 # unavailable.
80 else
81 set timeout=5
82 fi
This will pause grub for 5 sec in my case.
@zcobol Thanks for sharing, I was able to trace down the differences. I was only installing kernel6.12 and did not include kernel6.12-modules-extra. When I installed both on a fresh image it was able to boot up successfully.
The issue is that installing kernel6.12 is not pulling kernel6.12-modules-extra also, and this package contains the vmw_pvscsi driver, the para-vitualized SCSI driver causing your storage devices not to work. So, installing both packages solves the issue!
The kernel6.12-modules-extra package should be a dependency:
[ec2-user@cernobyl ~]$ sudo dnf install kernel6.12
Last metadata expiration check: 0:24:31 ago on Thu Apr 3 22:38:26 2025.
Dependencies resolved.
========================================================================================================================
Package Architecture Version Repository Size
========================================================================================================================
Installing:
kernel6.12 x86_64 6.12.20-23.97.amzn2023 amazonlinux 35 M
Transaction Summary
========================================================================================================================
Install 1 Package
Total download size: 35 M
Installed size: 167 M
Is this ok [y/N]:
@ccassidy-vaisala you might want to reopen the bug and let AWS people fix the dependencies.
@zcobol an interesting note: EC2 boots just fine without the extra modules, so it must be a VM-specific issue.
For the AWS team: Is there a way to ensure that this modules-extra is installed for on-prem/VM use cases?
@ccassidy-vaisala that's correct! Without the extra modules package, booting stops when trying to load seed.iso. If you remove quiet from the boot menu there is a lot more info on the console. The EC2 instances don't need a virtual CD-ROM.
This is where the it stops:
No, the extras modules shouldn't be a dependency as we don't want to install them by default on EC2. The whole point of that package is to carry a bunch of modules that are not normally useful on EC2 (mostly only on-prem or on some corner-cases).
I can't think of a good way of using rpm dependencies to express the need to install kernel-modules-extras-6.12 is kernel-module-extras was installed already. We might just need to improve the documentation
Unfortunately, kernel-modules-extra for 6.12 is kernel6.12-modules-extra.
Best we could do so far is to add an FAQ in our public docs: https://docs.aws.amazon.com/linux/al2023/ug/kernel-update.html#al2023-kernel-faq
We are looking into making things simpler with a DNF module that would model the "if installed, use the new package on update" rule, but it might take a bit to sort this out.
We have updated our documentation at https://docs.aws.amazon.com/linux/al2023/ug/kernel-update.html#al2023-kernel-faq to make sure that for a 6.1 -> 6.12 upgrade folks also update their kernel-{headers,devel,libbpf,...} packages to the 6.12 equivalents. I'm assuming that following these instructions the VMware case is also covered. Resolving.