OpenBSD 7.4 unable to boot installer image

[0x1F680]

Hello, I am trying to create an OpenBSD vm. Here are my steps:

vm iso https://cdn.openbsd.org/pub/OpenBSD/7.4/amd64/install74.iso

(the template was changed - one line: grub_install0="kopenbsd -h com0 /7.4/amd64/bsd.rd")

vm create -t openbsd openbsd_vm
vm install -f openbsd install74.iso

It boots up fine with cd0, then it hangs for about 5-10sec (not sure hot to get verbosity), and then spontaneously reboots. After that the installer iso (cd0) unmoounts and naturally hd0,1 does not exist. The boot fails. I tried the method above with the 6.2 version of OpenBSD, that works... I am not sure how to resolve the problem. Have been picking around, but even manually loading the OpenBSD kernel in grub and trying to boot that way does not work.

[0x1F680]

I was also able to start the vm with the following config using the uefi mode and the install74.img installer:

loader="uefi"
cpu=4
memory=5G
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="install74.img"
disk1_type="virtio-blk"
disk1_name="disk0.img"
bhyve_options="-w"
uuid="79ae551c-78ff-11ee-8f23-08bfb876c95c"
network0_mac="58:9c:fc:08:98:f3"

But I am getting the following error:

cannot open hd0a:/etc/random.seed: No such file or directory
booting hd0a:/7.4/amd64/bsd.rd: 3969732+1655808+3886664+0+708608 [109+444888+297417]=0xa76798
entry point at 0x1001000
wrmsr to register 0xc0011029(0x3) on vcpu 0

[stephen-fox]

fwiw, there is a known issue starting with OpenBSD 7.4 where (supposedly) grub kopenbsd no longer works:

• https://bsd.network/@brynet/111245178032951793
• https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273814

It sounds like UEFI boot should work - but I am not sure which options will make vm-bhyve happy. From my discussions with another user (who I believe is using just bhyvectl):

• install OpenBSD using grub but select EFI (GPT) partitioning (you can tell you did it because an “i” partition appears)
• at the reboot (end of install) halt
• shut down the VM
• edit /bhyve/machinename/machinename.conf changing it from grub to uefi (yes, just that line, ignore the rest)
• boot and it will work, console too

• https://bsd.network/@cynicalsecurity/111262678468763422

I have not had any luck myself trying to get it to boot using UEFI.

[0x1F680]

Update:

So far I have realized that for some reason openbsd does not support com0 output and when you set com0 as the default output the boot is unsuccessful (error above). Ended up using vnc, and that worked. Am going to try some more options. Config:

loader="uefi"
cpu=4
memory=5G
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="install74.img"
disk1_type="virtio-blk"
disk1_name="disk0.img"
bhyve_options="-w"
uuid="79ae551c-78ff-11ee-8f23-08bfb876c95c"
network0_mac="58:9c:fc:08:98:f3"
graphics="yes"
xhci_mouse="yes"
graphics_res="1600x900"
graphics_wait="yes"
utctime="YES"

[0x1F680]

#531

[stephen-fox]

I think it is worth leaving this issue open so others can easily find it until upstream fixes it. I do not think a graphical boot is a viable work around in all cases.

[0x1F680]

You are right! Thanks, for some reason I did not think of that...

[stephen-fox]

I've been playing with this some more and it looks like booting with a .iso does not work when the vm-bhyve loader value is uefi.

tl;dr - it appears OpenBSD does not support booting UEFI from an .iso. However, serial console output works just fine if you execute set tty com0 at the boot> prompt. According to user qilo on reddit (https://www.reddit.com/r/openbsd/comments/ckviqs/comment/jkklz2h):

cd73.iso and install73.iso won't work, because OpenBSD's (U)EFI bootloader BOOTX64.EFI expects the kernel to be found on FFS filesystem/partition, not on optical media filesystem ISO 9660

I tried out your patch (#531) and removed all the graphics stuff and loaded the .img OpenBSD installer as a second disk (as you are doing). With this configuration I see output on the serial console - but the installer fails to boot unless you execute set tty com0 at the boot> prompt. Once the tty value is set, I can install 7.4 successfully.

Here is my vm-bhyve configuration file for reference:

loader="uefi"
cpu=1
memory=256M
network0_type="virtio-net"
network0_switch="your_network"
disk0_type="virtio-blk"
disk0_name="disk0.img"
disk1_type="virtio-blk"
disk1_name="install74.img"
bhyve_options="-w"

If I comment out disk1_type and disk1_name and run vm install openbsd-7.4-test openbsd-install74-amd64.iso, I never get any serial console output. If I do the same thing but I enable graphics, this is what I see in the VNC session:

... so it never even boots the OpenBSD installer.

[0x1F680]

The .iso installer does not work with UEFI because it does not contain the UEFI boot files (they are included in the .img installer though).

OpenBSD has removed support for the legacy console standard: https://github.com/Openbsd/src/commit/745c2f60e98fd1f418c104960a567e120624d705. I am pretty sure that grub2-bhyve can support it relatively easily, but for some reason does not want to work for me yet. That might be worth exploring. Haven't had the chance to play around enough...

Side note: vm img ${some_url} can download the .img images, I assume. Is there a way to use those downloads as the installer image with vm install ...?

[rtprio]

Side note: vm img ${some_url} can download the .img images, I assume. Is there a way to use those downloads as the installer image with vm install ...?

(I use curl or wget to fetch images into /vm/.iso. ) However, I can get to the openbsd installer no problem on UEFI with vm install router2 install74.img.
It seems you can also: vm iso https://cdn.openbsd.org/pub/OpenBSD/7.4/amd64/install74.img

However, I'm running into something else which prevents running OpenBSD any further

pvbus0 at mainbus0: bhyve                                                               
pci0 at mainbus0 bus 0                                                                  
0:4:0: io address conflict 0xc080/0x80     

[0x1F680]

I believe that when you run vm install router2 install74.img the install image gets mounted as an ahci-cd device and not as an virtio-blk device. That is the issue when using vm install.

[rtprio]

Either using vm install or adding disk1 as virtio-blk functions the same for me.

On Thu, Nov 9, 2023 at 4:21 PM 0x1F680 @.***> wrote:

I believe that when you run vm install router2 install74.img the install image gets mounted as an ahci-cd device and not as an virtio-blk device. That is the issue when using vm install.

— Reply to this email directly, view it on GitHub https://github.com/churchers/vm-bhyve/issues/530#issuecomment-1804974916, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG4ENNJKMDP5L4AJYSOPELYDWFSNAVCNFSM6AAAAAA6Z55Y2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBUHE3TIOJRGY . You are receiving this because you commented.Message ID: @.***>

[stephen-fox]

vm install example path/to/img works for me - but the VM appears to reboot with the .img still attached causing it to re-enter the installer. This is problematic for me because I am doing an unattended installation using my mkobsd tool. I worked around this by having the installer poweroff the VM when it finishes the installation. After restarting it with vm start example, it boots as expected.

For anyone interested, I just added support to mkobsd for the .img installer type. I also added support for customizing the installer tree (this makes it easy to automate adding set tty com0 to /etc/boot.conf). I included an example of creating a bhyve compatible UEFI installer in mkobsd's examples directory.

[arrigo]

• install OpenBSD using grub but select EFI (GPT) partitioning (you can tell you did it because an “i” partition appears)
• at the reboot (end of install) halt
• shut down the VM
• edit /bhyve/machinename/machinename.conf changing it from grub to uefi (yes, just that line, ignore the rest)
• boot and it will work, console too

• https://bsd.network/@cynicalsecurity/111262678468763422

I have not had any luck myself trying to get it to boot using UEFI.

For reference, as the author of the recipe above, this no longer works - you must install OpenBSD in UEFI mode using a "graphics" installer and VNC but set the console to serial. You can then remove the graphics and just boot using something like this:

loader="uefi"
cpu="2"
memory="8G"

This will give you a classic "server" set up with a serial console which will work with vm console and allow you to upgrade with sysupgrade without needing to turn on graphics & VNC again.

[rtprio]

That wasn't the problem either. My problem was the package edk2.

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274389

On Thu, Dec 28, 2023 at 8:41 AM Arrigo @.***> wrote:

• install OpenBSD using grub but select EFI (GPT) partitioning (you can tell you did it because an “i” partition appears)

• at the reboot (end of install) halt

• shut down the VM

• edit /bhyve/machinename/machinename.conf changing it from grub to uefi (yes, just that line, ignore the rest)

• boot and it will work, console too

• @.***/111262678468763422

I have not had any luck myself trying to get it to boot using UEFI.

For reference, as the author of the recipe above, this no longer works - you must install OpenBSD in UEFI mode using a "graphics" installer and VNC but set the console to serial. You can then remove the graphics and just boot using something like this:

loader="uefi" cpu="2" memory="8G"

This will give you a classic "server" set up with a serial console which will work with vm console and allow you to upgrade with sysupgrade without needing to turn on graphics & VNC again.

— Reply to this email directly, view it on GitHub https://github.com/churchers/vm-bhyve/issues/530#issuecomment-1871409222, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG4ENIFG3BY2FXT6GQGNV3YLW4ONAVCNFSM6AAAAAA6Z55Y2KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZRGQYDSMRSGI . You are receiving this because you commented.Message ID: @.***>