dockur
[Bug]: MacOS Sonoma crashes on boot with more than 1 CPU
Operating system
Ubuntu 24.04
Description
Running docker image 1.10 (see https://github.com/dockur/macos/issues/83) with 1 CPU I can get Sonoma installed and it seems to work. Changing to 4 or 2 CPUs however makes the masOS crash on boot. After changing back to 1 CPU the system starts again. The bug report macOS generates is attached.
Docker compose
services:
macos:
image: dockurr/macos
container_name: macos
environment:
VERSION: "sonoma"
RAM_SIZE: "10G"
CPU_CORES: "4"
DISK_SIZE: "256G"
devices:
- /dev/kvm
cap_add:
- NET_ADMIN
ports:
- 8006:8006
- 5900:5900/tcp
- 5900:5900/udp
stop_grace_period: 2m
volumes:
- /storage/osx:/storage
Docker log
docker compose logs
macos | ❯ Starting macOS for Docker v1.10...
macos | ❯ For support visit https://github.com/dockur/macos
macos | ❯ CPU: AMD Ryzen 5 4600G | RAM: 11/12 GB | DISK: 462 GB (ext4) | HOST: 6.8.0-45...
macos |
macos | ❯ Warning: unexpected clocksource: kvm-clock
macos | ❯ Warning: your configured RAM_SIZE of 10 GB is very close to the 11 GB of memory available, please consider a lower value.
macos | ❯ Booting macOS 1920x1080 using QEMU v9.0.2...
macos | qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]
macos | qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.erms [bit 9]
macos | qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10]
macos | qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
macos |
BdsDxe: failed to load Boot0080 "Mac OS X" from PciRoot(0x0)/Pci(0xA,0x0)/VenMedia(BE74FCF7-0B7C-49F3-9147-01F4042E6842,F407ECD909D6DF4288953882CAB087A7)/\B9ACD089-962F-4937-9CD0-F429E3B3CD41\System\Library\CoreServices\boot.efi: Not Found
macos | BdsDxe: failed to load Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0xA,0x0): Not Found
macos | BdsDxe: loading Boot0002 "UEFI Misc Device 2" from PciRoot(0x0)/Pci(0x5,0x0)
macos | BdsDxe: starting Boot0002 "UEFI Misc Device 2" from PciRoot(0x0)/Pci(0x5,0x0)
macos | #[EB|LOG:EXITBS:END] _
macos | #[EB.BST.FBS|-]
macos | #[EB|B:BOOT]
macos | #[EB|LOG:HANDOFF TO XNU] _
macos | ======== End of efiboot serial output. ========
MasOS bug report
panic(cpu 1 caller 0xffffff8001e54b4e): Non-monotonic time: invoke at 0x6ad95c891, runnable at 0x6ad95d658 @sched_prim.c:3031
Panicked task 0xffffff91fb327bd8: 5 threads: pid 147: cfprefsd
Backtrace (CPU 1), panicked thread: 0xffffffa06085c598, Frame : Return Address
0xffffff8a537f7950 : 0xffffff8001e265a1 mach_kernel : _handle_debugger_trap + 0x4b1
0xffffff8a537f79a0 : 0xffffff8001f8bcd0 mach_kernel : _kdp_i386_trap + 0x110
0xffffff8a537f79e0 : 0xffffff8001f7b28c mach_kernel : _kernel_trap + 0x55c
0xffffff8a537f7a60 : 0xffffff8001dc0971 mach_kernel : _return_from_trap + 0xc1
0xffffff8a537f7a80 : 0xffffff8001e2688d mach_kernel : _DebuggerTrapWithState + 0x5d
0xffffff8a537f7b70 : 0xffffff8001e25f33 mach_kernel : _panic_trap_to_debugger + 0x1e3
0xffffff8a537f7bd0 : 0xffffff80025db78b mach_kernel : _panic + 0x84
0xffffff8a537f7cc0 : 0xffffff8001e54b4e mach_kernel : _thread_block_reason + 0x21ce
0xffffff8a537f7d50 : 0xffffff8001e52a70 mach_kernel : _thread_block_reason + 0xf0
0xffffff8a537f7da0 : 0xffffff80022cbedf mach_kernel : _workq_thread_set_max_qos + 0x31bf
0xffffff8a537f7e10 : 0xffffff80022ca7b1 mach_kernel : _workq_thread_set_max_qos + 0x1a91
0xffffff8a537f7ec0 : 0xffffff80022cf1b9 mach_kernel : _workq_kernreturn + 0x489
0xffffff8a537f7f40 : 0xffffff8002455170 mach_kernel : _unix_syscall64 + 0x1f0
0xffffff8a537f7fa0 : 0xffffff8001dc0db6 mach_kernel : _hndl_unix_scall64 + 0x16
Process name corresponding to current thread (0xffffffa06085c598): cfprefsd
Boot args: keepsyms=1
Mac OS version:
23H124
Kernel version:
Darwin Kernel Version 23.6.0: Wed Jul 31 20:48:44 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_X86_64
Kernel UUID: 67760982-25DC-34FC-A09E-E5624B8B4D62
roots installed: 0
KernelCache slide: 0x0000000001a00000
KernelCache base: 0xffffff8001c00000
Kernel slide: 0x0000000001ae4000
Kernel text base: 0xffffff8001ce4000
__HIB text base: 0xffffff8001b00000
System model name: iMacPro1,1 (Mac-7BA5B2D9E42DDD94)
System shutdown begun: NO
Panic diags file available: NO (0xe00002bc)
Hibernation exit count: 0
System uptime in nanoseconds: 28682746701
Last Sleep: absolute base_tsc base_nano
Uptime : 0x00000006ad9ffe96
Sleep : 0x0000000000000000 0x0000000000000000 0x0000000000000000
Wake : 0x0000000000000000 0x0000000c097e2e4d 0x0000000000000000
Compressor Info: 0% of compressed pages limit (OK) and 0% of segments limit (OK) with 0 swapfiles and OK swap space
Zone info:
Zone map: 0xffffff8b93c26000 - 0xffffffab93c26000
. PGZ : 0xffffff8b93c26000 - 0xffffff8b95027000
. VM : 0xffffff8b95027000 - 0xffffff90619f3000
. RO : 0xffffff90619f3000 - 0xffffff91fb28c000
. GEN0 : 0xffffff91fb28c000 - 0xffffff96c7c58000
. GEN1 : 0xffffff96c7c58000 - 0xffffff9b94625000
. GEN2 : 0xffffff9b94625000 - 0xffffffa060ff2000
. GEN3 : 0xffffffa060ff2000 - 0xffffffa52d9bf000
. DATA : 0xffffffa52d9bf000 - 0xffffffab93c26000
Metadata: 0xffffff8a31616000 - 0xffffff8a51616000
Bitmaps : 0xffffff8a51616000 - 0xffffff8a53416000
Extra : 0 - 0
last started kext at 17045942211: @filesystems.autofs 3.0 (addr 0xffffff7f973ff000, size 32768)
loaded kexts:
com.khronokernel.CryptexFixup 1.0.1
as.acidanthera.BlueToolFixup 2.6.9
as.acidanthera.BrcmFirmwareStore 2.6.9
as.vit9696.!AALC 1.9.0
as.vit9696.WhateverGreen 1.6.7
as.vit9696.Lilu 1.6.8
@filesystems.autofs 3.0
>AGPM 135
>X86PlatformShim 1.0.0
>!APlatformEnabler 2.7.0d0
>!AGraphicsDevicePolicy 8.1.9
>!AUpstreamUserClient 3.6.11
>!AMCCSControl 1.17
@AGDCPluginDisplayMetrics 8.1.9
>pmtelemetry 1
@filesystems.nfs 1
|IOUserEthernet 1.0.1
>usb.!UUserHCI 1
>!AUIO 1
>!ASunrise 1
>!AHV 1
>!A!ISlowAdaptiveClocking 4.0.0
>!A!IMCEReporter 115
>!A16X50ACPI 3.2
>ACPI_SMC_PlatformPlugin 1.0.0
>!AFIVRDriver 4.1.0
>!AFileSystemDriver 3.0.1
@filesystems.tmpfs 1
@filesystems.lifs 1
@filesystems.apfs 2236.141.1
@filesystems.hfs.kext 650.140.2
@BootCache 40
@!AFSCompression.!AFSCompressionTypeZlib 1.0.0
@!AFSCompression.!AFSCompressionTypeDataless 1.0.0d1
>!AAHCIPort 380
>!AACPIButtons 6.1
@private.KextAudit 1.0
>!ARTC 2.0.1
>!ASMBIOS 2.1
>!AAPIC 1.7
@!ASystemPolicy 2.0.0
@nke.applicationfirewall 405
|IOKitRegistryCompatibility 1
|EndpointSecurity 1
@Dont_Steal_Mac_OS_X 7.0.0
>!ADiskImages2 276.120.7
@kec.!AEncryptedArchive 1
>!AGraphicsControl 8.1.9
>!ASMBus!C 1.0.18d1
|IOSMBus!F 1.1
>!ASMBusPCI 1.0.14d1
>!UAudio 640.12
|IOAudio!F 540.3
@vecLib.kext 1.2.0
>!AAudioClockLibs 340.8
@kext.triggers 1.0
>IOHIDPowerSource 1
>!ACallbackPowerSource 1
|IOAVB!F 1220.1
>!ARSMChannel 1
|IORSM!F 1
>!AIPAppender 1.0
@!AGPUWrangler 8.1.9
@!AGraphicsDeviceControl 8.1.9
|IOSlowAdaptiveClocking!F 1.0.0
@plugin.IOgPTPPlugin 1240.15
>!A16X50Serial 3.2
>IOPlatformPluginLegacy 1.0.0
>X86PlatformPlugin 1.0.0
>IOPlatformPlugin!F 6.0.0d8
>driverkit.serial 6.0.0
|IONDRVSupport 598
|IOGraphics!F 598
>usb.IOUSBHostHIDDevice 1.2
>usb.cdc 5.0.0
>usb.networking 5.0.0
>usb.!UHostCompositeDevice 1.2
>!AXsanScheme 3
>!ABSDKextStarter 3
|IOSurface 352.50.1
@filesystems.hfs.encodings.kext 1
>!ASyntheticGame!C 11.6.1
>usb.!UHostPacketFilter 1.0
|IOUSB!F 900.4.2
|IOAHCI!F 304
>!AVirtIO 221.100.2
|IOSerial!F 11
>usb.!UXHCIPCI 1.2
>usb.!UXHCI 1.2
|IOHID!F 2.0.0
>!AEFINVRAM 2.1
>!AEFIRuntime 2.1
|IOTimeSync!F 1240.15
|IOSkywalk!F 1.0
>mDNSOffloadUserClient 1.0.1b8
|IONetworking!F 3.4
>DiskImages 493.0.0
|IO!B!F 9.0.0
|IOReport!F 47
$quarantine 4
$sandbox 300.0
@kext.!AMatch 1.0.0d1
>!ASSE 1.0
>!ALockdownMode 1
>!AKeyStore 2
>!UTDM 556
|IOUSBMass!SDriver 245
|IOUSBHost!F 1.2
>!UHostMergeProperties 1.2
>usb.!UCommon 1.0
>!ABusPower!C 1.0
|IOSCSIBlockCommandsDevice 495
|IOSCSIArchitectureModel!F 495
>!AFDEKeyStore 28.30
>!AEffaceable!S 1.0
|IO!S!F 2.1
>!ACyrus 1
>!AMobileFileIntegrity 1.0.5
$!AImage4 6.3.0
@kext.CoreTrust 1
>!ACredentialManager 1.0
|CoreAnalytics!F 1
>!ASEPManager 1.0.1
>IOSlaveProcessor 1
>!AACPIPlatform 6.1
>!ASMC 3.1.9
|IOPCI!F 2.9
|IOACPI!F 1.4
>watchdog 1
@kec.pthread 1
@kec.XrtHostedXnu 1
@kec.Libm 1
@kec.Compression 1.0
@kec.corecrypto 14.0
Screenshots (optional)
No response
I have no clue why this happens. Only solution I can offer right now is to use an older version, like Ventura.
I have no clue why this happens. Only solution I can offer right now is to use an older version, like Ventura.
Also happened on ventura
Its not so useful to just hear "happens to me too", it would be better to include docker logfile like @jbramburek did.
That way we can try to track down under which circumstances this happens. For example, do you all have an "AMD Ryzen 5 4600G" processor like @jbramburek or do you all have a weird clocksource (@jbramburek has kvm-clock which is very strange, like he uses nested virtualization?), etc. Only if we discover what you guys all share in common, we can figure out what is causing this.
- Docker logs almost look the same
- I am using nested virtualization with AMD EPYC 7B13 and AMD Ryzen 9 5950X. Both have the same issues and identical logs.
@urfavteddybear So you get the same message:
unexpected clocksource: kvm-clock
?
Exactly. It happens on Sequoia, Sonoma, Ventura, and Monterey. I haven't tried Big Sur yet, though it seems fine when running on 1 CPU.
I created a new version now (v1.14) that will automaticly detect nested virtualization and restricts the container to a single core when the default of 2 was selected. To prevent this problem for new users.
But ofcourse this doesnt solve the root problem, that macOS crashes when using nested virtualization on AMD in combination with multiple cores.
However Im afraid I will not be able to solve it. Either it is a problem in macOS, or in OpenCore or in QEMU/KVM.
Is there any good reason why you run this container with nested virtualization? I never expected anyone to do so. Because if you already have a way to run VM's there seems no need for a container that can run a VM?
I am running this in a VM on my Proxmox server. I haven't really explored other options to get this os so far.
@jbramburek I would advice to install macOS directly in Proxmox then, instead of using this container. This project is more for people that do not have Proxmox but just Docker.
Somehow I can make it work on AMD EPYC 7443P but not with Ryzen series.
EDIT: SOLVED
Having this issue on
Linux 6.8.0-49-generic #49~22.04.1-Ubuntu x86_64 x86_64 x86_64 GNU/Linux
After reading this thread I already started it with 1 CPU
docker run -it --rm -v ./volumes/:/storage -e CPU_CORES=1 -p 8006:8006 -p 5900:5900/tcp -p 5900:5900/udp --device=/dev/kvm --device=/dev/net/tun --cap-add NET_ADMIN --stop-timeout 120 dockurr/macos
but I still get this error.
Image verification complete!
❯ Creating a 64G growable disk image in raw format...
❯ Building boot image...
❯ Booting macOS using QEMU v9.1.1...
... a lot of empty lines until this comes
BdsDxe: failed to load Boot0001 "UEFI Misc Device" from PciRoot(0x0)/Pci(0xA,0x0): Not Found
BdsDxe: loading Boot0002 "UEFI Misc Device 2" from PciRoot(0x0)/Pci(0x5,0x0)
BdsDxe: starting Boot0002 "UEFI Misc Device 2" from PciRoot(0x0)/Pci(0x5,0x0)
#[EB|LOG:EXITBS:END] _
#[EB.BST.FBS|-]
#[EB|B:BOOT]
#[EB|LOG:HANDOFF TO XNU] _
======== End of efiboot serial output. ========
$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 46 bits physical, 48 bits virtual Byte Order: Little Endian CPU(s): 16 On-line CPU(s) list: 0-15 Vendor ID: GenuineIntel Model name: 12th Gen Intel(R) Core(TM) i7-1270P CPU family: 6 Model: 154 Thread(s) per core: 2 Core(s) per socket: 12 Socket(s): 1 Stepping: 3
and if I test my virtualisation I get:
$ sudo kvm-ok INFO: /dev/kvm exists KVM acceleration can be used
Please let me know, if you need more information
@joergi This limitation with 1 core is only for (certain) AMD CPU's, because macOS expects only Intel CPU's. But you have an Intel CPU, so there should be no need to restrict it to 1 core, and your issue is most likely not related at all.
@joergi so there should be no need to restrict it to 1 core, and your issue is most likely not related at all.
okay. I had the same issue, when I had no cores defined...
The error comes always, BUT if I connect via the browser, it's working! nice