Nested Virtualization

[ghost]

trafficstars

add +vmx to -cpu args from : -cpu Penryn,vendor=GenuineIntel,kvm=on,+sse3,+sse4.2,+aes,+xsave,+avx,+xsaveopt,+xsavec,+xgetbv1,+avx2,+bmi2,+smep,+bmi1,+fma,+movbe,+invtsc
to : -cpu Penryn,vendor=GenuineIntel,kvm=on,+sse3,+sse4.2,+aes,+xsave,+avx,+xsaveopt,+xsavec,+xgetbv1,+avx2,+bmi2,+smep,+bmi1,+fma,+movbe,+invtsc,+vmx \

to make Nested Virtualization like vm in vm.

[ddavidebor]

Does this work correctly with docker, virtualbox, parallels etc etc?

[ghost]

yes,it works with HAXM : https://github.com/intel/haxm

[Pilo11]

This is working perfectly but only with Intel CPUs :) With AMD CPUs it'd be +svm, but this is useless because Intel HAXM is'nt compatible with AMD virtualization :)

Does anyone know how an alternative way?

[halykon]

Just did this and VMX is enabled on the guest. Xcode Simulator works perfectly fine, yet neither Parallels nor VMware Fusion is working. VMware shows an "Unsupported CPU detected" error, while Parallels doesn't start at all. Do I have to reinstall macOS when changing something about the CPU configuration?

[XertroV]

This is working perfectly but only with Intel CPUs :) With AMD CPUs it'd be +svm, but this is useless because Intel HAXM is'nt compatible with AMD virtualization :)

I don't think you can with AMD -- I looked into it but couldn't get it working (even with kvm_amd nested=1 etc).

Also, AMD gets HAXM like acceleration for free now: https://developer.android.com/studio/run/emulator-acceleration (helps with android at least)

[Joseph-Hui]

It didn't work for me. Docker didn't run. What will it return if you run sysctl kern.hv_support?

[elovin]

Maybe you have to hide the fact that the os is running inside a virtual machine from the os, this is also required if you want to run windows 10 with nested virtualisation on kvm so that docker for windows works.

[volkertb]

@Pilo11 There is a port of KVM to Windows called GVM, which specifically supports the hardware-assisted virtualization features of AMD CPUs: https://github.com/google/android-emulator-hypervisor-driver-for-amd-processors

Could that help as an alternative for HAXM here?

[halykon]

So to anyone still wondering: You can pass the +vmx flag which works for programs not using the macOS hypervisor (VirtualBox for example iirc). Anything using the macOS built in hypervisor will not work as it is detecting the emulated Penryn CPU (ignoring SMBIOS). You can use nested virtualization if you change the bootloader from Clover to OpenCore, because OpenCore supports Host CPU passthrough.

[niklasgrewe]

@halykon thanks for this declaration. That's what i need. Tow questions about that...

1. How can add the vmx flag (i am using the virtmanager ui)
2. How can i change the bootloader from Clover to OpenCore?

[halykon]

Hi there @niklasgrewe, you can pass the vmx flag in virt-manager by enabeling xml-editing in the settings and then editing the cpu line in the xml file (it's on the very bottom).

Switching the bootloader from clover to opencore is a bit trickier, for the time being i switched to proxmox because there is a premade EFI by Nick Sherlok on his GitHhub, however i'm not really happy with proxmox so i'll probably make my own EFI (or try to get his running without proxmox). I'll let you know on my progress and i'll probably do a pull request once it's finished so people can choose between clover and opencore.

[kourindouhime]

works using OpenCore and host instead of Penryn in cpu string (q35, ovmf) on 440fx worked with Penryn and clover (mbr, seabios)

[EduFrazao]

There is no alternative for AMD Processor folks?

[512yottabytes]

I'm running MacOs in KVM inside Docker on AMD Ryzen 7. All works fine.

пт, 9 квіт. 2021, 05:03 користувач Eduardo Frazão @.***> пише:

There is no alternative for AMD Processor folks?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/foxlet/macOS-Simple-KVM/issues/142#issuecomment-816348547, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDKSSMT24M6Y4YHIKJRLPDTHZN7ZANCNFSM4JQ5NDOA .

[Pilo11]

I'm running MacOs in KVM inside Docker on AMD Ryzen 7. All works fine. пт, 9 квіт. 2021, 05:03 користувач Eduardo Frazão @.***> пише: … There is no alternative for AMD Processor folks? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#142 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDKSSMT24M6Y4YHIKJRLPDTHZN7ZANCNFSM4JQ5NDOA .

That's fine but we discussed regarding NESTED virtualization. So it is not possible to set up an OS "sub"-guest within the MacOS KVM guest because MacOS does not officially support AMD CPUs and it's SVM stuff (only Intels VT-x). At least if we use the PENRYN CPU emulation...

[bazuka5801]

I'm running on Intel, HAXM test utils works fine

~/Downloads/checktool
CPU vendor          *  GenuineIntel
Intel64 supported   *  Yes
VMX supported       *  Yes
VMX enabled         *  Yes
EPT supported       *  Yes
NX supported        *  Yes
NX enabled          *  Yes
Hyper-V disabled    *  Not Applicable
OS version          *  macOS 12.6.1
OS architecture     *  x86_64
Guest unoccupied    *  Yes. 0 guest(s)

But android emulator not starting with kernel error

[    0.044000] tsc: Unable to calibrate against PIT
[    0.048000] tsc: HPET/PMTIMER calibration failed
[    0.056000] tsc: Marking TSC unstable due to could not calculate TSC khz
[    0.060000] Calibrating delay loop...
[    0.064000] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[    0.064000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.41-android13-8-00055-g4f5025129fe8-ab8949913 #1
[    0.064000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
[    0.064000] RIP: 0010:delay_halt_tpause+0x11/0x20
[    0.064000] Code: b8 f0 00 00 00 b9 02 00 00 00 0f 01 fb 5b 5d c3 00 00 cc cc 00 00 cc cc 00 55 48 89 e5 48 8d 04 37 48 89 c2 48 c1 ea 20 31 c9 <66> 0f ae f1 5d c3 00 00 cc cc 00 00 cc cc 00 55 48 89 e5 48 8d 3c
[    0.064000] RSP: 0000:ffffffffa5e03e00 EFLAGS: 00010246
[    0.064000] RAX: 0000075ac0ee8df5 RBX: 0000000000001000 RCX: 0000000000000000
[    0.064000] RDX: 000000000000075a RSI: 0000000000001000 RDI: 0000075ac0ee7df5
[    0.064000] RBP: ffffffffa5e03e00 R08: 0000000000000000 R09: ffffffffa5e6b5d0
[    0.064000] R10: 00000000ffffefff R11: ffffffffa3d75a60 R12: 0000000000001000
[    0.064000] R13: 0000000000000001 R14: 0000075ac0ee7df5 R15: 0000000000000001
[    0.064000] FS:  0000000000000000(0000) GS:ffff99c32b200000(0000) knlGS:0000000000000000
[    0.064000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.064000] CR2: ffff99c31de01000 CR3: 000000005d80c000 CR4: 00000000000406b0
[    0.064000] Call Trace:
[    0.064000]  <TASK>
[    0.064000]  delay_halt+0x32/0x60
[    0.064000]  calibrate_delay+0x60b/0x750
[    0.064000]  ? static_key_slow_inc+0x79/0x1c0
[    0.064000]  start_kernel+0x401/0x5ab
[    0.064000]  ? early_idt_handler_array+0x120/0x120
[    0.064000]  x86_64_start_reservations+0x24/0x26
[    0.064000]  x86_64_start_kernel+0x107/0x110
[    0.064000]  secondary_startup_64_no_verify+0xc3/0xcb
[    0.064000]  </TASK>
[    0.064000] Modules linked in:
[    0.068000] ---[ end trace cf2f24958332d59f ]---
[    0.072000] RIP: 0010:delay_halt_tpause+0x11/0x20
[    0.076000] Code: b8 f0 00 00 00 b9 02 00 00 00 0f 01 fb 5b 5d c3 00 00 cc cc 00 00 cc cc 00 55 48 89 e5 48 8d 04 37 48 89 c2 48 c1 ea 20 31 c9 <66> 0f ae f1 5d c3 00 00 cc cc 00 00 cc cc 00 55 48 89 e5 48 8d 3c
[    0.080000] RSP: 0000:ffffffffa5e03e00 EFLAGS: 00010246
[    0.084000] RAX: 0000075ac0ee8df5 RBX: 0000000000001000 RCX: 0000000000000000
[    0.088000] RDX: 000000000000075a RSI: 0000000000001000 RDI: 0000075ac0ee7df5
[    0.092000] RBP: ffffffffa5e03e00 R08: 0000000000000000 R09: ffffffffa5e6b5d0
[    0.096000] R10: 00000000ffffefff R11: ffffffffa3d75a60 R12: 0000000000001000
[    0.100000] R13: 0000000000000001 R14: 0000075ac0ee7df5 R15: 0000000000000001
[    0.104000] FS:  0000000000000000(0000) GS:ffff99c32b200000(0000) knlGS:0000000000000000
[    0.108000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.112000] CR2: ffff99c31de01000 CR3: 000000005d80c000 CR4: 00000000000406b0
[    0.116000] Kernel panic - not syncing: Fatal exception
[    0.116000] invalid opcode: 0000 [#2] PREEMPT SMP PTI
[    0.116000] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D           5.15.41-android13-8-00055-g4f5025129fe8-ab8949913 #1

[HMaker]

I'm running MacOs in KVM inside Docker on AMD Ryzen 7. All works fine. пт, 9 квіт. 2021, 05:03 користувач Eduardo Frazão @.***> пише: … There is no alternative for AMD Processor folks? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#142 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDKSSMT24M6Y4YHIKJRLPDTHZN7ZANCNFSM4JQ5NDOA .

That's fine but we discussed regarding NESTED virtualization. So it is not possible to set up an OS "sub"-guest within the MacOS KVM guest because MacOS does not officially support AMD CPUs and it's SVM stuff (only Intels VT-x). At least if we use the PENRYN CPU emulation...

you found any way to speed up nested virtualization with AMD CPUs? The MacOS is smooth but iOS Simulator for example is unusable.