v86 icon indicating copy to clipboard operation
v86 copied to clipboard

XV6 fails with "ide disk 1 not present". (Does v86 emulate IDE?)

Open Rodmatronic opened this issue 3 months ago • 17 comments

Version: ccb5e1ce (Sep 20, 2025 11:09) Firefox 142 on Linux Mint 22.2

XV6 does not initialize past the panic iderw: ide disk 1 not present.

//PAGEBREAK!
// Sync buf with disk.
// If B_DIRTY is set, write buf to disk, clear B_DIRTY, set B_VALID.
// Else if B_VALID is not set, read buf from disk, set B_VALID.
void
iderw(struct buf *b)
{
  struct buf **pp;

  if(!holdingsleep(&b->lock))
    panic("iderw: buf not locked");
  if((b->flags & (B_VALID|B_DIRTY)) == B_VALID)
    panic("iderw: nothing to do");
  if(b->dev != 0 && !havedisk1)
    panic("iderw: ide disk 1 not present");

Compiled from the XV6 X86 Github repo: https://github.com/mit-pdos/xv6-public

Rodmatronic avatar Sep 21 '25 23:09 Rodmatronic

Yes, V86 supports IDE drives (ATA and ATAPI), but it's not perfect.

Building the image failed for me, could you provide your image file to download? I'd like to take a look.

chschnell avatar Sep 23 '25 11:09 chschnell

I will provide the built image in a few hours, I am currently occupied and do not have access to the image.

Rodmatronic avatar Sep 23 '25 13:09 Rodmatronic

According to the XV6 sources of ide.c the error message you get is a failed attempt to access the 2nd hard disk of the primary IDE controller. This disk would be hdb in V86, have you tried booting with something mounted to hdb?

chschnell avatar Sep 23 '25 18:09 chschnell

This disk would be hdb in V86, have you tried booting with something mounted to hdb?

I also think so, according to https://cshung.github.io/posts/xv6-on-v86/ (https://github.com/copy/v86/pull/354) and https://github.com/mit-pdos/xv6-public/blob/master/Makefile#L222, xv6 should be mounted with two hard disks images, the first is xv6.img and the second is fs.img (from the build files).

SuperMaxusa avatar Sep 23 '25 18:09 SuperMaxusa

Alright, after much debugging, it fails to read blocks from the disk (even with FS.IMG in the slave slot) I will upload my compiled images shortly, and do further testing. Before I tried XV6, I had actually tried my XV6 derived kernel, and it had this exact issue. It would freeze when reading a block, waiting for the request to finish.

If it is not this issue, then it might be something to do with MP support.

//PAGEBREAK!
// Sync buf with disk.
// If B_DIRTY is set, write buf to disk, clear B_DIRTY, set B_VALID.
// Else if B_VALID is not set, read buf from disk, set B_VALID.
void
iderw(struct buf *b)
{
  struct buf **pp;

  if(!holdingsleep(&b->lock))
    panic("iderw: buf not locked");
  if((b->flags & (B_VALID|B_DIRTY)) == B_VALID)
    panic("iderw: nothing to do");
  if(b->dev != 0 && !havedisk1)
    panic("iderw: ide disk 1 not present");

  acquire(&idelock);  //DOC:acquire-lock

  // Append b to idequeue.
  b->qnext = 0;
  for(pp=&idequeue; *pp; pp=&(*pp)->qnext)  //DOC:insert-queue
    ;
  *pp = b;

  // Start disk if necessary.
  if(idequeue == b)
    idestart(b);

  // Wait for request to finish.

  // hangs here on v86
  while((b->flags & (B_VALID|B_DIRTY)) != B_VALID){
    sleep(b, &idelock);
  }


  release(&idelock);
}

Rodmatronic avatar Sep 23 '25 19:09 Rodmatronic

I uploaded my compiled source compressed on this repo: https://github.com/Rodmatronic/compiledxv6

Rodmatronic avatar Sep 23 '25 19:09 Rodmatronic

Turns out, I failed to give it 512 MB of ram which it requires. (was using the default 128 MB.) Seems to be a v86 issue with IDE support. This shouldn't happen, normally XV6 gracefully crashes at low memory. New issue?

Image

Rodmatronic avatar Sep 23 '25 19:09 Rodmatronic

I got a lot of APIC logs from the debug log in https://copy.sh/v86/debug.html:

v86_all_debug.js:1 22:35:45+358 [CPU ] Not delivered: irr already set, vector=20
v86_all_debug.js:1 Previous message repeated 238 times
v86_all_debug.js:1 22:35:47+767 [CPU ] warning: APIC skipping 1 interrupts initial=10000000 ticks=29659999 last_tick=128987.9ms now=129017.6ms d=29.7ms
v86_all_debug.js:1 22:35:47+769 [CPU ] Not delivered: irr already set, vector=20
v86_all_debug.js:1 Previous message repeated 143 times
v86_all_debug.js:1 22:35:49+224 [CPU ] warning: APIC skipping 1 interrupts initial=10000000 ticks=26955000 last_tick=130447.6ms now=130474.5ms d=27.0ms
v86_all_debug.js:1 22:35:49+225 [CPU ] Not delivered: irr already set, vector=20

UPD: the 0x20 vector is Local APIC that is not implemented in v86 (I guess?), thanks to @cshung.

SuperMaxusa avatar Sep 23 '25 19:09 SuperMaxusa

This disk would be hdb in V86, have you tried booting with something mounted to hdb?

I also think so, according to https://cshung.github.io/posts/xv6-on-v86/ (#354) and https://github.com/mit-pdos/xv6-public/blob/master/Makefile#L222, xv6 should be mounted with two hard disks images, the first is xv6.img and the second is fs.img (from the build files).

The 2020 tutorial from @cshung is an interesting read.

I think it needs two patches for XV6 from there (3301652 and 1a229ce) in order to make this work.

chschnell avatar Sep 24 '25 11:09 chschnell

Thank you for your interest in trying out. As you all might notice, the work was quite a bit dated, so good luck bringing it back up.

Just like any work that is not maintained, it naturally goes stale and fail, since it worked back in the days, we might be able to bisect from there.

May I ask what is the goal here? If it aligns with my personal objective (i.e. to learn about operating system and virtual machine internals), I might be able to participate.

Here are a couple more investigation blogs from me in this area for your references.

FYI @orniv

cshung avatar Sep 24 '25 15:09 cshung

@cshung Thanks for the response!

5 years is certainly a long time, but only V86 has changed while XV6 stayed the same which should reduce the problem of bringing it back up again, correct?

At this point I just have a few beginner questions:

  • When was the last time you had this working?
  • Is https://github.com/cshung/xv6-public/tree/public/dev/andrewau/v86-support the correct branch of XV6 for V86?
  • Which toolchain did you use to compile the sources, does it need a 32-bit compiler?
  • What's the correct RAM setting, 224MB or 240MB?
  • Run it with or without APIC support in V86?

I'm using gcc 12.2.0 under Debian Amd64 (and I have serious doubts about that, I think I should cross-compile), I had to add this to Makefile to build the two images:

CFLAGS += -Wno-infinite-recursion -Wno-array-bounds

I hopefully find some time this weekend to take a deeper look at this.

May I ask what is the goal here? If it aligns with my personal objective (i.e. to learn about operating system and virtual machine internals), I might be able to participate.

I'd say the goal is to bring XV6 back up again. I believe that those that actively invest time and contribute to V86 are in for the learning, but I can only speak for myself.

And also thanks for your blog posts, which I find equally interesting.

chschnell avatar Sep 24 '25 17:09 chschnell

Build from andrewau/xv6-public/v86-support: xv6-v86.tar.gz

I used Tiny Core Linux 7.2 with gcc 5.2 to build. In v86, set 512 MB RAM and enable ACPI. Currently it stops at assertion failed: pic.master.get_irq().is_none():

21:42:26+809 [CPU ] ocw3: a v86_all_debug.js:1:767
21:42:26+809 [CPU ] ocw3: 68 v86_all_debug.js:1:767
21:42:26+809 [CPU ] special mask mode: true v86_all_debug.js:1:767
21:42:26+809 [CPU ] ocw3: a v86_all_debug.js:1:767
21:42:26+810 [    ] SERIAL: xv6... v86_all_debug.js:1:767
21:42:27+124 [CPU ] Overlapping first=[from=103a4e to=103a64 is_entry=1] second=[from=103a60 to=103a64 is_entry=0] v86_all_debug.js:1:767
21:42:27+166 [CPU ] Finished compiling for page at 102000 v86_all_debug.js:1:767
21:42:27+562 [    ] SERIAL: cpu0: starting 0 v86_all_debug.js:1:767
panicked at src/rust/cpu/pic.rs:323:5:
assertion failed: pic.master.get_irq().is_none() v86_all_debug.js:239:112
Uncaught RuntimeError: unreachable executed
    re https://copy.sh/v86/build/v86_all_debug.js:258
    onmessage https://copy.sh/v86/build/v86_all_debug.js:261
v86_all_debug.js line 240 > WebAssembly.instantiate:31465:1
21:42:27+578 [CPU ] Finished compiling for page at 106000 v86_all_debug.js:1:767
21:42:27+579 [CPU ] unused after overwrite 897 v86_all_debug.js:1:767

SuperMaxusa avatar Sep 24 '25 19:09 SuperMaxusa

Run it with or without APIC support in V86?

Probably without back then. But I recently pushed some improvements for the apic, and it passes tests across all OSes. So we could run xv6 with apic and remove those hacks.

Currently it stops at assertion failed: pic.master.get_irq().is_none():

I think this assertion is incorrect, could you remove it and test again?

copy avatar Sep 24 '25 20:09 copy

I think this assertion is incorrect, could you remove it and test again?

Both versions of xv6 hangs on this screen.

Debug log from patched xv6: xv6-log-new.txt Debug log from unpatched xv6: xv6-log-new2.txt

SuperMaxusa avatar Sep 24 '25 22:09 SuperMaxusa

With my setup I also get this output when it hangs:

SeaBIOS (version rel-1.16.2-0-gea1b7a0)
Booting from Hard Disk...
cpu0: starting 0
sb: size 0 nblocks 0 ninodes 0 nlog 0 logstart 0 inodestart 0 bmap start 0
lapicid 0: panic: ilock: no type
 80101897 80101e99 80102113 80100b07 8010572a 80104c49 80105c7d 80105a13 0 0

The sb: size 0 nblocks 0 ... stems from iinit() in fs.c, sb means "superblock" in XV6 and this is where it reads the superblock from fs.img into RAM. The superblock in XV6 is the second block on the device (with the usual sector size of 512 bytes). Looking at struct superblock it becomes obvious that it is an error to have all fields set to zero (as the output shows). In my case, the superblock is definitely not all zeros:

$ hexdump -s512 -n512 xv6-fs.img
0000200 03e8 0000 03ad 0000 00c8 0000 001e 0000
0000210 0002 0000 0020 0000 003a 0000 0000 0000
0000220 0000 0000 0000 0000 0000 0000 0000 0000
*
0000400

See attached xv6-trace-disk.txt for a full trace log of the V86 IDE controller when booting XV6. An overview over this log:

lines 1-5: V86 IDE controller initialization (loads images into RAM)
lines 6-190: XV6 probes all 4 IDE devices (ide0.0, ide0.1, ide1.0 and ide1.1)

line 190: read block 0x01 from ide0.0 (note: HDA)
line 206: read block 0x02 from ide0.0
...
line 1598: read block 0x5A from ide0.0
line 1614: read block 0x5B from ide0.0 (0x5B = dec 91, 23.552 KB)

line 1637: switch to ide0.1 (note: HDB)
line 1638: read blocks 0x0...0x100 from ide0.1
line 1657: read block 0x0 from ide0.1
line 1678: write block 0x0 to ide0.1

To summarize, hda works (obviously, how else would we boot XV6 this far) and hdb fails with XV6. I had tested hdb with various OSes when I worked on ide.js, but it seems not enough. I'm still trying to make sense of it.

chschnell avatar Sep 25 '25 09:09 chschnell

Tonight I am able to get the system to work locally, with a big caveat that it is running all the old code.

On the Xv6 side, I need to disable -Werror as the new toolchain is finding new warnings that the old code has. The key is to use the public/dev/andrewau/v86-support branch on my fork at and to use make DEFINE='-D HACK_FOR_V86' xv6.img fs.img to build so that we actually ignore the old broken APIC support.

On the v86 side, there is no need to make any changes on top of the private/fix-ps2-a20 branch on my fork.

Since the Xv6 side is basically stale code, there is probably nothing to investigate there.

On the v86 side, the project changed a lot. If I were to investigate, I would try to bisect the error from the history, since we know it used to work in the past. I don't have the time right now for that, maybe I can this weekend, maybe ...

I imagine this is going to be fairly tedious to nail it down. It would be awesome we can get some automation for that.

cshung avatar Sep 26 '25 03:09 cshung

It works using Bochs BIOS, the only problem seems to be SeaBIOS here. Note that V86 is unmodified.

Image

XV6 build setup I used:

  • Debian 12 Amd64
  • gcc 12.2.0 (64 bit)
  • XV6 source branch: https://github.com/cshung/xv6-public/tree/public/dev/andrewau/v86-support
  • XV6 Makefile: add CFLAGS += -Wno-infinite-recursion -Wno-array-bounds
  • Tested with or without ACPI, ~240MB RAM (didn't test any other RAM size)~ EDIT: 512MB RAM

See here for a live demo at my demo site: https://www.rekoba.de/~cs/v86/demo?p=xv6


EDIT:

I checked which patches to XV6 are still required with the current V86 (and Bochs BIOS), and it turns out that only a single patch is needed, the one that disables SMP and switches back to the uniprocessor model.

So I forked mit-pdos/xv6-public, applied that patch and only modified the Makefile to suppress warnings "infinite-recursion" and "array-bounds", see here for the fork: https://github.com/chschnell/xv6-public/tree/v86

To build simply run make xv6.img fs.img

Archive with both image files: xv6-v86.zip


EDIT 2: XV6 running at copy.sh

https://github.com/user-attachments/assets/a2e6dfd2-59ac-4fe4-ab00-1534c1734c02

chschnell avatar Sep 27 '25 04:09 chschnell