Support host/guest pkgs distinction for make-disk-image

[felixscheinost]

If I am not missing something it is currently not possible to use different system for QEMU creating the disk image vs the pkgs used in the image, right?

This means I really need to be on the same architecture as otherwise double virtualisation happens which is really slow.

When trying to cross-build e.g. aarch64-linux on x86_64-linux, the QEMU is aarch64-linux.

When using binfmt to emulate that, QEMU itself is already running emulated, then emulating another machine so double emulation, right?

NixOS last year or so introduced the ability to override the pkgs used for QEMU using virtualisation.host.pkgs. This way I can even run a x86_64-linux on aarch64-darwin using acceptable speed!

Would it be possible to add something like that?

I see that there's disko.imageBuilder.pkgs but I think that isn't what I want as it is used for both QEMU and e.g. dependencies used inside the VM.

Would you be open to a PR trying to tackle that?

Maybe would it even make sense to instead of nixpkgs' runInLinuxVM try to use NixOS's virtualisation infrastructure which already supports this? Or was something like that already attempted?

[iFreilicht]

I'm not sure I understand the issue 100%, but wouldn't it be enough to set disko.imageBuilder.qemu? If so, we can definitely change the default value of that to use qemu from buildPackages.

[felixscheinost]

I think just disko.imageBuilder.qemu isn't enough but may be 50% :)

It starts that the script isn't executable on the host system as writeCheckedBash uses ${pkgs.bash}/bin/bash.

To be honest I haven't worked with nixpkgs cross-compilation enough.

[felixscheinost]

I think this may be related to this issue https://github.com/NixOS/nixpkgs/issues/73470

[felixscheinost]

So seems this really isn't possible without reusing or re-implementing what NixOS virtualisation.host.pkgs has done.

[felixscheinost]

For anyone interested: Using this branch (has a dependant nixpkgs PR) I can successfully build Linux disk images on Darwin, quite fast as well. https://github.com/felixscheinost/disko/tree/runInLinuxVM-darwin.

I'd keep this issue open for a while if that's ok? I'm expecting that the nixpkgs PR of mine won't be accepted as is but maybe the necessary changes can be made upstream sometime.