Haiku OS guest support

[LekKit]

Milestones, progress

• [x] Out-of-tree Haiku boots using M-mode haiku_loader under RVVM with ATA drive
• [x] Nightly Haiku EFI boots through U-Boot with NVMe (Unstable framebuffer, app_server crashes, NVMe in polling mode)
• [x] Input devices with upstream Haiku drivers support (See #58)
• [x] Network card with upstream Haiku drivers support (RTL8169, but need stable tap_user first)
• [x] Verify other utility devices (RTC, syscon) work. Are goldfish or DS1742 RTCs supported in Haiku?
• [x] Fix NVMe IRQ loss with Haiku EFI (Doesn't happen in QEMU)
• [x] Fix Haiku EFI bootloader crashing with new U-Boot efifb (Somehow works now, weird fluke?...)
• [x] Fix Haiku EFI bootloader crashing in SMP (Doesn't happen in QEMU)
• [x] NVMe support in M-Mode haiku_loader
• [x] Fix display server crashing upon late userspace init in Haiku EFI (Unrelated to RVVM?)

[LekKit]

@X547 As far as I see, NVMe IRQ loss doesn't happen when running your Haiku build with ATA & haiku_loader.riscv. It doesn't timeout or anything, ATA & NVMe coexist happily, both drive partitions are enumerated, no complaints about NVMe polling, etc. Perhaps there are some non-upstream changes that fix it. There is a small issue with haiku_loader.riscv however. If ATA isn't the first device on the PCI bus, it crashes, i.e. NVMe cannot be attached before ATA right now.

Another interesting thing, EFI framebuffer seems to work properly now with nightly images & new U-Boot. I don't know what fixed it, I tried older RVVM commits but it still works... Maybe it was just a fluke or some local issue, huh.

Would be happy if you can verify these.

u-boot.bin.zip

[X547]

If ATA isn't the first device on the PCI bus, it crashes, i.e. NVMe cannot be attached before ATA right now.

ATA MMIO address is currently hardcoded both in boot loader and kernel. Kernel ATA driver need refactor because it currently assumes that register addresses are 16 bit.

[LekKit]

If ATA isn't the first device on the PCI bus, it crashes, i.e. NVMe cannot be attached before ATA right now.

ATA MMIO address is currently hardcoded both in boot loader and kernel. Kernel ATA driver need refactor because it currently assumes that register addresses are 16 bit.

I hope we can just ignore all of this and get NVMe running instead. Where can I find haiku_loader.riscv sources? I could try writing a simple NVMe driver, why not (Fine deal imo, since you're working on I2C HID).

[X547]

Where can I find haiku_loader.riscv sources? I could try writing a simple NVMe driver, why not (Fine deal imo, since you're working on I2C HID).

It is here: https://github.com/haiku/haiku/blob/master/src/system/boot/platform/riscv/devices.cpp#L33. It is needed to add some NvmeBlockDevice. I have an unpublished boot loader PCI bus code.

[X547]

My Haiku RVVM branch: https://github.com/X547/haiku/tree/rvvm2.

WIP NVMe boot loader driver: https://github.com/X547/haiku/blob/e717045595ebbd71a30731bc57c96a5d1a68ef52/src/system/boot/platform/riscv/NvmeBlockDevice.cpp.

[X547]

ATA MMIO address hardcode:

• https://github.com/X547/haiku/blob/e717045595ebbd71a30731bc57c96a5d1a68ef52/src/system/boot/platform/riscv/AtaBlockDevice.cpp#L16
• https://github.com/X547/haiku/blob/e717045595ebbd71a30731bc57c96a5d1a68ef52/src/add-ons/kernel/generic/ata_adapter/ata_adapter.cpp#L54

[LekKit]

I'm not sure I properly understand how to build Haiku ../../buildtools/jam/jam0 -j16 -q @minimum-mmc

Asked for riscv target boot platform 
Unknown path to handle adding to image 
don't know how to make @minimal-mmc
...patience...
...found 1 target(s)...
...can't find 1 target(s)...

[X547]

I'm not sure I properly understand how to build Haiku

You need to configure build first. It will build GCC for riscv64 target. Assuming that current directory contains haiku and buildtools.

mkdir -p generated.riscv64
cd generated.riscv64
../configure -j4 --build-cross-tools riscv64 --cross-tools-source ../../buildtools --distro-compatibility official

[X547]

../../buildtools/jam/jam0 -j16 -q @minimum-mmc don't know how to make @minimal-mmc

Spell miss? Correct is @minimum-mmc.

[LekKit]

You need to configure build first

I did that already using this guide https://www.haiku-os.org/guides/building/compiling-riscv64

Spell miss? Correct is @minimum-mmc.

I tried many, none worked (With the same error)

[LekKit]

Improved upstream ATA in 43aeba3, at least it's no longer a security hellhole (3 CWEs fixed, lol). Merged your API changes so you no longer need to hack on it each time.

Is it worth adding some kind of -ata option for those drives in upstream?

[X547]

I tried many, none worked (With the same error)

https://www.haiku-os.org/guides/building/pre-reqs

<jam-install-command>

To install jam you can use one of two commands: The first requires administrative privilege, as jam will be installed to ‘/usr/local/bin/’

    sudo ./jam0 install
    ./jam0 -sBINDIR=$HOME/bin install

[X547]

Is it worth adding some kind of -ata option for those drives in upstream?

Ideally it will be nice to have an option to specify drive type for each image independently.

[LekKit]

Is it worth adding some kind of -ata option for those drives in upstream?

Ideally it will be nice to have an option to specify drive type for each image independently.

Yes, it's just a convention that -i/-image means "Just give me any kind of storage that is preferred". ATA is kind of deprecated because I see little use for it in context of a RISC-V system (Outside of Haiku bootloader, and even this is temporary), and because it isn't maintained well. It's not like I'm against this device, but no one is gonna implement missing features / non-critical fixes for it any more. I only ran a bit of fuzzing/coverage because I don't want to put my users under security risk for using it, and because someone had to do it.

I have no plans for more storage devices currently. That's why I don't know what should I do with the CLI interface, really.

[X547]

Did you solve a problem of Haiku build? What Haiku source version are you using? What happens if run jam @minimum-raw kernel?

[LekKit]

Did you solve a problem of Haiku build? What Haiku source version are you using? What happens if run jam @minimum-raw kernel?

Using your Haiku fork, rvvm2 branch Figured the jam issue, thanks. There are some compilation errors tho

../src/system/boot/platform/efi/arch/riscv64/arch_traps.cpp: In function 'void WriteSstatus(uint64_t)':
../src/system/boot/platform/efi/arch/riscv64/arch_traps.cpp:50:30: error: no matching function for call to 'SstatusReg::SstatusReg(uint64_t&)'
   50 |         SstatusReg status(val);
      |                              ^

[X547]

Figured the jam issue, thanks. There are some compilation errors tho

Fixed, source updated.

[X547]

Functional configuration:

• RVVM: https://github.com/X547/RVVM/commit/9579da6ad23cbbe95ecaec178d4a33a24dd0dc2b
• Haiku: https://github.com/X547/haiku/commit/dc7f6642ec87bfaa4d9731a91e57fdbf0c0e7cc0
• Haiku build: jam -q -j4 @minimum-raw haiku-minimum.image
• Run: rvvm -mem 512M -res 1024x768 --rv64 objects/haiku/riscv64/release/system/boot/riscv/haiku_loader.riscv --image haiku-minimum.image

[LekKit]

Functional configuration:

Hurray, I now can at least try it since we have working i2c hid and stuff... Feels great (Tho perf could be better, I'm currently losing to QEMU here perhaps. Haiku uses floats a lot, right?)

Will proceed to NVMe bootloader driver

[LekKit]

Hmm, sometimes I2C HID deadlocks apparently. Pretty rare to spot but I've seen these 2 times already.

WARN: Possible deadlock at src/devices/hid-mouse.c@97
WARN: The lock was previously held at src/devices/hid-mouse.c@213
WARN: Version: RVVM v0.5-8e8f200-git
WARN: Attempting to recover execution...
 * * * * * * *

WARN: Possible deadlock at src/devices/i2c-hid.c@155
WARN: The lock was previously held at src/devices/i2c-hid.c@318
WARN: Version: RVVM v0.5-8e8f200-git
WARN: Attempting to recover execution...
 * * * * * * *

[LekKit]

Verify other utility devices (RTC, syscon) work. Are goldfish or DS1742 RTCs supported in Haiku?

Syscon doesn't seem to work. Powering off the system from the guest leaves me with some win2000-vibe message "It's now safe to turn off the computer" and the machine never actually powers down.

Should be trivial to implement, syscon is just a single mmio register with specific values for poweroff/reset. This is also used in QEMU and on SiFive boards AFAIK.

[X547]

Haiku currently support shutdown and RTC with HTIF commands. RTC HTIF interface is my extension and it work only in my TinyEMU fork.

[LekKit]

Haiku currently support shutdown and RTC with HTIF commands. RTC HTIF interface is my extension and it work only in my TinyEMU fork.

I can implement that as well probably?

[X547]

I can implement that as well probably?

I think that it is better to implement more standard interfaces.

HTIF commands currently used by Haiku:

• (device: 0, cmd: 0, arg: 1): shutdown.
• (device: 1, cmd: 1, arg: ch): print char ch.
• (device: 2, cmd: 0, arg: 0): get UNIX time in microseconds.

Executing HTIF command:

// host-target interface
struct HtifRegs
{
	uint32 toHostLo;
	uint32 toHostHi;
	uint32 fromHostLo;
	uint32 fromHostHi;
};

uint64
HtifCmd(uint32 device, uint8 cmd, uint32 arg)
{
	if (gHtifRegs == 0)
		return 0;

	uint64 htifTohost = ((uint64)device << 56)
		+ ((uint64)cmd << 48) + arg;
	gHtifRegs->toHostLo = htifTohost % ((uint64)1 << 32);
	gHtifRegs->toHostHi = htifTohost / ((uint64)1 << 32);
	return (uint64)gHtifRegs->fromHostLo
		+ ((uint64)gHtifRegs->fromHostHi << 32);
}

FDT compatible value: ucb,htif0.

[LekKit]

I think that it is better to implement more standard interfaces.

Yeah, syscon and goldfish rtc were implemented just because they match basic QEMU machine. I would prefer emulating hardware from real RV boards (current PLIC/CLINT/UART/I2C-OC/NVMe fall into this category well) or just some generic common hardware (simple-fb perhaps counts?.. My RPi uses this driver as well). As I see HTIF is a basic interface for debugging FPGA boards, right? That's great if it's part of official spec, I've just never seen it for some reason.

[LekKit]

Haiku dd reports incorrect transfer speed (0.0 or -0.0!), crashes the kernel. No meaningful backtrace, perhaps I should build Haiku with -fno-omit-frame-pointer. As for "does it happen outside RVVM" I dunno, should be checked soon. Values -0.0 suspiciously look like some FPU-related trouble. Don't mind the terrible read speeds, my host laptop HDD is really that slow

vm_page_fault: vm_soft_fault returned error 'Bad address' on fault at 0x303231353237313e, ip 0x3eabcd2226, write 0, user 1, exec 0, thread 0x1a8
PANIC: thread_hit_serious_debug_event
Welcome to Kernel Debugging Land...
Thread 424 "dd" running on CPU 0
Stack:
FP: 0x0
kdebug> bt
Stack:
FP: 0xffffffc006664870
FP: 0xffffffc005c58298, PC: 0xffffffc0000ca8a6 <kernel_riscv64> invoke_debugger_command.localalias + 170
FP: 0x0, PC: 0x100000040 0x100000040
kdebug> 

[X547]

PANIC: thread_hit_serious_debug_event

This is userland process crash. It can be continued with es command. This kernel panic usually don't happen. It is temporary added here (https://github.com/X547/haiku/blob/dc7f6642ec87bfaa4d9731a91e57fdbf0c0e7cc0/src/system/kernel/debug/user_debugger.cpp#L816) for debugging purposes.

dd seems miscompiled. Crash happends on all RISC-V platforms (all supported emulators and real hardware).

[LekKit]

dd seems miscompiled. Crash happends on all RISC-V platforms (all supported emulators and real hardware).

Hmm, alright then. The 0.0/-0.0 values too?

[X547]

The 0.0/-0.0 values too?

It is probably related to crash cause. Need to check dd source code (it is a part of GNU coreutils).

[LekKit]

Interesting note: Running without RVJIT doesn't affect guest Haiku responsiveness / CPU consumption much. I'm pretty surprised, because it feels super smooth even though RVVM interpreter should emulate CPU with speeds about 300-500 MHz on my host.

Am I right that Haiku uses floats extensively? Currently any FPU code forces RVVM to cross-jump between JIT/interpreter, which is not ideal. Perhaps with further FPU JIT support this could be significantly faster.