Can't start VM from VirtualBox UI with hardening enabled

[dionorgua]

Bug Description

I can't start VM from VirtualBox UI if compiled without --disable-hardening.

At the same time I can start VM using CLI:

VirtualBoxVM --driverless --startvm VM_NAME

--driverless here is important.

I'm 100% sure that I'm launching just compiled binaries.

Also rebuilding package with --disable-hardening fixes it (it's also needed to remove the SUID bit from the VirtualBoxVM binary)

How to Reproduce

Compile without --disable-hardening:

        ./configure \
            --ose \
            --odir=$(abspath out) \
            --with-linux="/usr" \
            --with-kvm \
            --disable-kmods \
            --enable-webservice \
            --enable-vde \
            --enable-vnc

Note: I've compiled it as debian package by using standard virtualbox_7.0.14-dfsg-4 with added KVM patch

VM configuration

Host OS details:

• Debian Unstable
• 6.7.9-amd64

Logs

No VM logs are created

[tpressure]

I don't have a good way to test this right now, but I poked around a little bit and came up with this:

0004-XXX-fix-hardened-builds-with-KVM.patch.txt

It's a long shot, but feel free to try.

[dionorgua]

Surely. Will test. I've used much bigger patch locally that modifies fDriverless in all places where CLI args are parsed.

[dionorgua]

@tpressure your patch works!

[tpressure]

@dionorgua that's amazing. I will double check that this is the best way to solve this and add it to our next release.

[eugenesan]

I can confirm the patch solved the issue. Thanks @tpressure !

I've created a package for Ubuntu (should also work on Debian) and it works as expected. If you want test it, you can find it here: https://launchpad.net/~eugenesan/+archive/ubuntu/boiler/+packages?field.name_filter=virtualbox&field.status_filter=published&field.series_filter=noble

[tpressure]

@eugenesan That's super cool. Are you actively maintaining this PPA or is this just for personal use? I'm asking because I am planning to compile a list of packages so people can find them more easily.

[eugenesan]

@tpressure I uploaded the package to my testing PPA as a proof of concept.

It should be trivial to create a dedicated PPA and maintain it there.

Though, since this package replaces existing Virtualbox package we should decide if that is how it should be distributed or a new package that can co-exists with regular Virtualbox should be created.

Maybe we should loop-in Debian Virtualbox (https://tracker.debian.org/pkg/virtualbox) maintainers and ask if they have suggestions or even desire to maintain it.

[dionorgua]

@tpressure is it possible to use patched virtualbox-kvm in old way with vbox*.ko kernel modules?

If so, this will simplify packaging a lot (it will be enough to have just one universal version)

[tpressure]

@dionorgua the patched virtualbox with KVM backend does not care whether the kernel drivers are installed or not. They should never open /dev/vboxdrv and use /dev/kvm instead. This is at least true when building with --disable-hardening.

Looking at my patch above, the story with hardening enabled is proabably a little different. Right now, it would try to first open /dev/vboxdrv and only fall back to KVM if vboxdrv is not available. We probably want another solution where the backend never tries to open /dev/vboxdrv at all.

Let me get back to you.

[tpressure]

@dionorgua can you try this instead: 0004-XXX-fix-hardened-builds-with-KVM.patch.txt

The downside of this approach is that users still have to recompile the kernel modules on every kernel update which are no longer required, and get DKMS failures if the modules don't compile against the recent kernel. VBox will still work, but I think it would be better to not install and load the modules to begin with.

[tpressure]

@eugenesan @dionorgua can you please test the new patch and report back whether this also works? If it does, I would include it in the next release.

[eugenesan]

I tried to create a package supporting both KVM and VBoxDrv but that approach doesn't work. The idea was to make kernel modules package (DKMS) optional and when not installed default to KVM and when installed use Vboxdrv.

With 0004-XXX-fix-hardened-builds-with-KVM.patch.txt patch, vboxnet* modules weren't installed. I modified the patch by removing:

# KVM doesn't need the VirtualBox Ring 0 drivers
VBOX_WITH_VBOXDRV=
VBOX_WITH_NETFLT=
VBOX_WITH_NETFLT_CROSSBOW=
VBOX_WITH_NETADP=

and was able to build and install the package.

Unfortunately, even if vbox* modules were loaded. Virtualbox defaults to KVM virtualization without an option to switch to vboxdrv.

The question is: is it possible at all to switch between virtualization methods in the same binary?

For reference: Package: https://launchpad.net/~eugenesan/+archive/ubuntu/boiler/+sourcepub/16158291/+listing-archive-extra Patch: 91-fix-hardened-builds-with-KVM-n3.patch.txt BuildLog: https://launchpad.net/~eugenesan/+archive/ubuntu/boiler/+build/28496204/+files/buildlog_ubuntu-noble-amd64.virtualbox_7.0.18-dfsg-1.01~kvm~eugenesan~noble4_BUILDING.txt.gz

[tpressure]

@eugenesan I think I misunderstood your question. What I thought you were asking is whether it is possible to run the virtualbox KVM backend with the vbox drivers still installed.

What you were really asking is whether you can build a unified binary that can run both the old vboxdrv code, and the new NEM KVM code. This is not possible because VBOX_WITH_KVM is (and currently has to be) a compile time switch. The reason is that the NEM interface does not abstract interrupt virtualization. Interrupt virtualization differs a lot between vanilla VBox and VBox with KVM, so we had to introduce hard switches in the code, e.g. [1].

So, to summarize:

• you can use the KVM backend even if the vbox* drivers are installed, but then VirtualBox won't use them
• it's not possible to have a unified binary that can run legacy VirtualBox and VirtualBox with KVM at the same time

Unfortunately, even if vbox* modules were loaded. Virtualbox defaults to KVM virtualization without an option to switch to vboxdrv.

This sounds about right. It's the best we can do right now. I will add this patch to the next release.

Side note: it would of course be possible to introduce a proper interrupt virtualization interface to the NEM, but this would require a good amount of design and engineering effort. Right now, we unfortunately can't do such big tasks without some legitimate business interest.

I think the best way going forward right now would be to provide 2 individual packages and let the user decide whether to use vanilla vbox, or virtualbox with KVM. For the KVM variant, it does not matter if the drivers are installed or not, so at least packaging should be easier, at least if I understood you correctly.

[1] https://github.com/cyberus-technology/virtualbox-kvm/blob/dev/patches/0001-NEM-Implement-KVM-backend.patch#L977

[eugenesan]

@tpressure Thank you for explaining. In this case, it's better we use the 2nd version of the patch from https://github.com/cyberus-technology/virtualbox-kvm/issues/23#issuecomment-2127637051.

P.S. If both patches are applied but --with-kvm is NOT added to ./configure, will vboxdrv functionality work? Trying to figure if we at least can use the same source package.

[tpressure]

The patches are mutually exclusive, I would not recommend applying both.

I will add https://github.com/cyberus-technology/virtualbox-kvm/issues/23#issuecomment-2127637051 to the next release (in a few days).

[eugenesan]

The patches are mutually exclusive, I would not recommend applying both.

I will add #23 (comment) to the next release (in a few days).

Thanks. What I meant is if the main KVM patch and the patch from https://github.com/cyberus-technology/virtualbox-kvm/issues/23#issuecomment-2127637051 are applied, can we still build a regular version of virtualbox without --with-kvm?

[tpressure]

Thanks. What I meant is if the main KVM patch and the patch from #23 (comment) are applied, can we still build a regular version of virtualbox without --with-kvm?

Absolutely. Everything we did is wrapped inside VBOX_WITH_KVM which is only set when you build with --with-kvm.

[eugenesan]

Thanks. What I meant is if the main KVM patch and the patch from #23 (comment) are applied, can we still build a regular version of virtualbox without --with-kvm?

Absolutely. Everything we did is wrapped inside VBOX_WITH_KVM which is only set when you build with --with-kvm.

Excellent! In that case I'll attempt creating "split binaries" package. I'll post updates and possible questions in a new issue since it's out of the scope for this issue.

[tpressure]

@eugenesan the patch is now part of the latest release.

[eugenesan]

@eugenesan the patch is now part of the latest release.

Ubuntu/Debian package with a new (dev-20240617) version of the patch can be found here: https://launchpad.net/~eugenesan/+archive/ubuntu/boiler/+build/28585722

Unfortunately, I was unsuccessful in building an universal (kvm+vboxdrv) package. There were a few issues that would require considerable investment to resolve. So for now we'll have to deal with two separate packages.

[tpressure]

@eugenesan thanks for the new package. I think we can close this issue now, can we?

[dionorgua]

@tpressure Latest release works for me without any extra patches.

[eugenesan]

@tpressure Yes, I believe the issue is solved. Thank you for your help.

[tpressure]

@eugenesan @dionorgua thank your for your feedback. Closed.