test386.asm
test386.asm copied to clipboard
barotto
Test every non tested opcode
A comment from VOGONS thread https://www.vogons.org/viewtopic.php?f=9&t=60095:
XLAT instructions (both 16-bit and 32-bit versions) were faulty: it was checking memory against a write(for the memory operand) instead of a proper read.
Currently left(ignoring those still pending in the other issue(pushf(d)/popf(d)), as well as protected-mode instructions and related instruction for protected mode-specific functionality(0F beginning range instructions)):
- Logical operations in the 00-3F range.
- Logical operations in the 80-83 range.
- INC reg in the 41-47 range.
- DEC reg in the 49-4F range.
- String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers).
- TEST instructions(opcode 84-85).
- XCHG instructions(opcode 86-87).
- MOV instructions(opcode 88-8B).
- XCHG instructions(opcode 90-97).
- SAHF/LAHF instructions (opcode 9E-9F).
- MOV with immediate offset(opcodes A0-A3).
- 32-bit address-size string instructions(B/W/D).
- TEST AL/AX/EAX,imm(opcodes A8&A9).
- Different variants of SHLD/SHRD with operand-sizes.
- MOV reg,imm (opcodes B0-BF).
- RET near (opcode C2).
- MOV r/m,imm(opcodes C6-C7).
- ENTER/LEAVE(opcodes C6&C7).
- XLAT (opcode D7).
- LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3).
- IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
- JMP instructions(opcodes E9-EB).
- REP/REPE/REPNE (opcodes F0-F3).
- Opcode F6/F7 TEST r/m,imm
- Basic flag operations (opcodes F8-FD).
- INC r/m8/16/32 instructions(opcodes FE-FF).
- 32-bit relative jumps(0F80-0F8F instruction range).
Side note: opcode 82h is missing from the intel-opcodes list, which is an alias for opcode 80h( http://ref.x86asm.net/coder32.html#x82 ).
Just created a little 'testsuite' that executes many of the basic instructions mentioned in my previous post(although it requires to be logged and verified manually by looking for register/memory reads/writes/changes): https://bitbucket.org/superfury/unipcemu/src/9c54c037466f0e079e76a35987584b0687d42af8/UniPCemu/assembly/?at=master
Although I've coded opcodes 00-3F by using raw binary statements to make sure the nasm assembler doesn't actually create opcodes outside of that range somehow(2-byte versions of them).
The only test that actually verifies itself using assembly instructions is the ret imm testing, much in the way your stack tests run.
Edit: https://bitbucket.org/superfury/unipcemu/src/6c46742934fc8ffba70de4e139671244164270e9/UniPCemu/assembly/?at=master
Fixed some bugs in the basic *D opcodes for the 00-3F range being the wrong opcode byte.
So, combining both testsuite results, that only leaves possible errors in:
- Logical operations in the 80-83 range**.
- String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers).
- TEST instructions(opcode 84-85).
- MOV instructions(opcode 88-8B).
- SAHF/LAHF instructions (opcode 9E-9F).
- 32-bit address-size string instructions(B/W/D).
- Different variants of SHLD/SHRD with operand-sizes.
- MOV r/m,imm(opcodes C6-C7).
- ENTER/LEAVE(opcodes C8&C9).
- XLAT (opcode D7).
- LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3)**.
- IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
- JMP instructions(opcodes E9-EB).
- REP/REPE/REPNE (opcodes F0-F3).
- Basic flag operations (opcodes F8-FD).
- INC r/m8/16/32 instructions(opcodes FE-FF).
- 32-bit relative jumps(0F80-0F8F instruction range).
** partially tested and verified in both testsuites
Just found a little 'bug' in my emulator(which seems to be only half documented in the 80386 programmer's reference manual's far return instruction description). (E)SP is increased before popping SS:(E)SP during a stack switch to an outer privilege level(resulting CS.RPL>CPL), which is documented, but also after popping the SS:(E)SP(thus, increasing(popping) the caller's stack variables. The latter isn't documented within the 80386 programmer's reference manual as far as I can see.
Having fixed the far return using the immediate on both stack locations(both on source and destination stacks), the Extended Memory Tester v3.0 now properly detects the video card and extended memory, continuing to test memory:D
This app now runs properly: https://archive.org/details/msdos_TESTEXT3_shareware
Hmmmm..... One basic device that can be used to test the ISNS/OUTS instructions is the IDE/ATA harddisk's buffer(Command E4 to read the buffer, Command E8 to write the buffer). First fill it with a pattern manually(512 bytes), then read it back and verify the pattern. That should be able to verify all in(s)/out(s) instructions.
Edit: Although I know some of them work properly(8-bit plain, 8-bit string and 16-bit string) work already, since they're used during POST(8-bit plain), during disk reads(8-bit or 16-bit reads), disk writes(8-bit or 16-bit writes) and CD-ROM access(16-bit reads and writes). The only one of those still untested are string and normal 32-bit variants and normal 16/32-bit single input/output.
I see you're busy on some more (protected mode) tests. Great!
That now just leaves:
- String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers) and hard disk buffer(using the read/write buffer commands).
- MOV instructions(opcode 88-8B).
- SAHF/LAHF instructions (opcode 9E-9F).
- MOV r/m,imm(opcodes C6-C7).
- ENTER/LEAVE(opcodes C8&C9).
- XLAT (opcode D7).
- LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3)**.
- IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
- JMP instructions(opcodes E9-EB).
- Basic flag operations (opcodes F8-FD).
- 32-bit relative jumps(0F80-0F8F instruction range).
So mostly various kinds of move instructions and miscellaneous instructions.
And of course all remaining protected-mode functionality itself(task switching etc.) is still left to test.
I'm currently wondering if protected-mode functionality of UniPCemu still has bugs(concerning call gates and interrupts etc. pushing data on the stack with(out) stack switch) in various cases.
I've found a pretty big bug on my emulator comparing its memory dump with Bochs' after the execution of the test suite: the Dirty bit of a PTE is not properly set on a write when the same page has been previously accessed by a read. So there are a lot of behavioural tests to be implemented beside the various opcodes...
I've just ran Bochs 2.6.9 and dumped the 640K memory. Then I did the same with UniPCemu's MMU functionality(running the debugger, press and hold circle and then tap square) when reaching the HLT at the end of the POST 0xFF(when reaching the HLT status).
It reveals there's quite a lot wrong, apparently:
00000401 C2 00
00000402 10 00
00000405 8E 00
000020B8 67 07
000023E0 27 07
000023E4 27 07
000023E8 27 07
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFB7 05 46
0001FFBB 05 32
0001FFBF 56 00
0001FFC2 00 80
0001FFC3 D3 00
0001FFC4 72 40
0001FFC7 58 DB
0001FFC8 73 FF
0001FFCB 38 00
0001FFCC AD 00
0001FFCF 0F 14
0001FFD0 AB 00
0001FFD1 00 FF
0001FFD2 00 FF
0001FFD3 01 00
0001FFD4 00 C0
0001FFD7 EB 05
0001FFD8 FF 00
0001FFDB 04 0D
0001FFDC AD 6F
0001FFDF 20 32
0001FFE2 80 00
0001FFE3 07 00
0001FFE4 08 00
0001FFE6 00 80
0001FFE7 94 00
0001FFE8 72 40
0001FFED FF 00
0001FFEE FF 00
0001FFEF 07 00
0001FFF0 08 01
0001FFF1 00 01
0001FFF3 08 84
0001FFF4 AD 00
0001FFF5 00 FF
0001FFF6 00 FF
0001FFF7 0D FF
0001FFF8 00 FF
0001FFF9 00 FF
0001FFFA 00 FF
0001FFFB 13 6E
0001FFFC 00 75
0001FFFD 00 57
0002E000 01 00
0002E001 FF 00
0002E002 FF 00
0002E003 FF 00
0009F003 00 50
So that means:
- First IDT entry has an issue.
- Page table entry issues? Seems to be the Accessed bit not being updated correctly?
- Stack overflow? (FFFC-FFFF)
- More stack issues? (1FFB7-1FFFD)
- Normal test issues? (2E000-2E003 and 9F003)
First IDT entry has an issue.
That's the IDT area, but it's not necessarily a problem with the IDT management, could be something that writes garbage at the wrong address.
Page table entry issues?
Possible.
Stack overflow? (FFFC-FFFF)
Maybe, or again you could be writing garbage to the wrong addresses (see below).
More stack issues? (1FFB7-1FFFD)
Maybe.
Normal test issues? (2E000-2E003 and 9F003)
Can't tell right now what the segment at 0x2e000 is used for, but dword at 0x9F000 should be 0x50465046, for some reason you're missing the last byte 0x50 at 0x9F003, or you have overwritten it with 00.
It seems like you have problems with the MMU.
When debugging the direct memory writes by the Paging unit, I see:
- Write to address 0x000023e0 value 0x000f8027.
- Write to address 0x000023e4 value 0x000f9027.
- Write to address 0x000023e8 value 0x000fa027.
- Write to address 0x000020b8 value 0x0002e067 (at step EE).
So the Paging unit requests the memory mapping to actually write the correct values there at some point. Maybe something else is somehow overwriting it? Edit: Or maybe there's an issue in the writeback to memory after said breakpoint... Hmmmm... Edit: Already confirmed that the 0x23E0 memory location receives and stores value 0x27. So it's another bug that's somehow overwriting this value? Edit: Just added those addresses hardcoded to the MMU unit. Let's see if they're overwritten with different values normally(thus not an general emulator bug). Edit: Looking directly in the capture of the physical RAM(with memory gaps having been removed when dumping the RAM), the values written are actually in RAM at their supposed locations?
Edit: Looking again at my own created logs(made using MinGW-w64 using the following script), from UniPCemu's ROM directory(which contains the Bochs dump as well as the ROMs):
cp "..\captures\memory.dat" "fullmemory.dat"
dd if=/dev/null of=fullmemory.dat bs=1 count=1 seek=655360
cmp -l fullmemory.dat bochs-640k-memdump.bin | gawk '{printf "%08X %02X %02X\n", ($1-1), strtonum(0$2), strtonum(0$3)}'>memory.cmp.txt
It copies the memory.dat to the ROM directory for processing, then edits it to become 640K large, finally executing the difference dump as per your documentation.
I thought that the left was the Bochs dump, while the right value was my own. Looking at it again(as well as reading your documentation on the results of the gawk command again), it's reversed.
So those three pages aren't supposed to be accessed(since bits 5(PTE&PDE)&6(PDE) are supposed to be cleared instead of set in the end of logging). But they're accessed anyways, which isn't supposed to happen? So those paged accesses on those locations are instructions addressing a wrong point in memory, or incorrect mapping somehow?
PTE at 0x23e0 is for the page at 0xF8000. Its Accessed bit should be 0, not 1 (byte at 0x23e0 should be 0x07 not 0x27). It seems like you're fetching the NOPs that are present in that area of the test code. Or maybe your MMU is updating the wrong PTE.
I also think the dword at 0x9F000 is interesting. See what is wrinting 00 at 0x9F003, overwriting the 0x50.
About 9F003, I see 0x50 being written at 10:0000C455. Then 23E0 written 0x27 at instruction 0010:7FFB. Then 23E4 written 0x27 at instruction 0010:8FFE. Then 23E8 written 0x27 at instruction 0010:98A8. Then 20B8 written 0x67 at instruction 0010:99F7.
The current testsuite lst used: test386.zip
Just found a bug that caused the highest written byte to memory to remain unlogged(it was storing size-1 in the memory usage variable, instead of size). It still needed to add 1 to the current address for detection of memory usage, which it didn't(used for logging purposes only). Since address 9F003 is the last byte in the memory capture, that one was missing and thus filled with zeroes by the dd-command. Edit: It's gone now:
00000401 C2 00
00000402 10 00
00000405 8E 00
000020B8 67 07
000023E0 27 07
000023E4 27 07
000023E8 27 07
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFB7 05 46
0001FFBB 05 32
0001FFBF 56 00
0001FFC2 00 80
0001FFC3 D3 00
0001FFC4 72 40
0001FFC7 58 DB
0001FFC8 73 FF
0001FFCB 38 00
0001FFCC AD 00
0001FFCF 0F 14
0001FFD0 AB 00
0001FFD1 00 FF
0001FFD2 00 FF
0001FFD3 01 00
0001FFD4 00 C0
0001FFD7 EB 05
0001FFD8 FF 00
0001FFDB 04 0D
0001FFDC AD 6F
0001FFDF 20 32
0001FFE2 80 00
0001FFE3 07 00
0001FFE4 08 00
0001FFE6 00 80
0001FFE7 94 00
0001FFE8 72 40
0001FFED FF 00
0001FFEE FF 00
0001FFEF 07 00
0001FFF0 08 01
0001FFF1 00 01
0001FFF3 08 84
0001FFF4 AD 00
0001FFF5 00 FF
0001FFF6 00 FF
0001FFF7 0D FF
0001FFF8 00 FF
0001FFF9 00 FF
0001FFFA 00 FF
0001FFFB 13 6E
0001FFFC 00 75
0001FFFD 00 57
0002E000 01 00
0002E001 FF 00
0002E002 FF 00
0002E003 FF 00
The first two are instruction fetches triggering the accessed being set during writeback of the Page tables. 0x23E8 is the memory operand of the MOV at said address, offset AD7C.
It's checking an instruction fetch operand there at 10:7FFB. It's checking a DWORD at that location. A part of said operand is at F8000. It's the byte after 0F85. The PDE is 000023E0. The PTE to write back is 0x000f8027. Edit: The test386.lst confirms the 0x8000 byte is the final byte of the instruction(the jne error instruction, which is a 0F85 imm32 instruction).
It's the final byte of line 15393 in my test386.lst file.
About 9F003, I see 0x50 being written at 10:0000C455.
~~That's ok, the problem is that your dump at POST FFh shows a 0x00.~~ resolved
Then 23E0 written 0x27 at instruction 0010:7FFB.
~~Pages 0xF8000-0xFA000 shouldn't be accessed.~~ they should
Then 20B8 written 0x67 at instruction 0010:99F7.
This seems correct? IT IS
You say that Paged memory at 0xF8000-FA000 shouldn't be accessed, but there's executable code there that's executing in my case(as well as being present in the test386.lst).
The test386.lst says this about said instruction:
1055 00007FFB 0F85FF3F0000 <2> jne error
It seems to be one of the arithmetic tests that's executed.
The entire block of nasm code:
1055 <1> testBittestWFlags btr, %1, %2, %3, %4
1055 <2>
1055 00007FBC 66B80100 <2> mov ax, %4
1055 00007FC0 6650 <2> push ax
1055 00007FC2 9D <2> popf
1055 00007FC3 66B80100 <2> mov ax, %2
1055 00007FC7 660FBAF001 <2> o16 %1 ax, %3
1055 00007FCC 9C <2> pushf
1055 00007FCD 6658 <2> pop ax
1055 00007FCF 6625D508 <2> and ax, PS_ARITH
1055 00007FD3 663D0008 <2> cmp ax, %5
1055 00007FD7 0F8523400000 <2> jne error
1055 <2>
1055 <2>
1055 00007FDD 66B80100 <2> mov ax, %4
1055 00007FE1 6650 <2> push ax
1055 00007FE3 9D <2> popf
1055 00007FE4 66B80100 <2> mov ax, %2
1055 00007FE8 66B90100 <2> mov cx, %3
1055 00007FEC 660FB3C8 <2> o16 %1 ax, cx
1055 00007FF0 9C <2> pushf
1055 00007FF1 6658 <2> pop ax
1055 00007FF3 6625D508 <2> and ax, PS_ARITH
1055 00007FF7 663D0008 <2> cmp ax, %5
1055 00007FFB 0F85FF3F0000 <2> jne error
That's bit_m.asm, row 85 being assembled. Edit: Looking further up the inclusion list, it's test386.asm row 1052. In test386.asm, I see it's part of the E0 undefined instruction tests.
I'm getting confused by all the edits. PLEASE verify that 2nd column are the values of your emu, 3rd column are Bochs' Because I'm starting to think your post shows the opposite...
I've recompiled the test386.asm source code(it uses the BOCHS version and 386-specific tests(POST E0) enabled).
As I said in my last post, the strange page (which has a PTE/PDE located at paged address F8000 and onwards) is part of the E0 tests that are executing and are present in the test386.lst generated by the nasm compiler.
This is the result of UniPCemu's memory dump(which is a direct dump of physical RAM, which is zero-padded to 640K) compared against the Bochs memory dump, as in your instructions at the test386.asm main code page(Readme).
00000401 C2 00
00000402 10 00
00000405 8E 00
000020B8 67 07
000023E0 27 07
000023E4 27 07
000023E8 27 07
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFB7 05 46
0001FFBB 05 32
0001FFBF 56 00
0001FFC2 00 80
0001FFC3 D3 00
0001FFC4 72 40
0001FFC7 58 DB
0001FFC8 73 FF
0001FFCB 38 00
0001FFCC AD 00
0001FFCF 0F 14
0001FFD0 AB 00
0001FFD1 00 FF
0001FFD2 00 FF
0001FFD3 01 00
0001FFD4 00 C0
0001FFD7 EB 05
0001FFD8 FF 00
0001FFDB 04 0D
0001FFDC AD 6F
0001FFDF 20 32
0001FFE2 80 00
0001FFE3 07 00
0001FFE4 08 00
0001FFE6 00 80
0001FFE7 94 00
0001FFE8 72 40
0001FFED FF 00
0001FFEE FF 00
0001FFEF 07 00
0001FFF0 08 01
0001FFF1 00 01
0001FFF3 08 84
0001FFF4 AD 00
0001FFF5 00 FF
0001FFF6 00 FF
0001FFF7 0D FF
0001FFF8 00 FF
0001FFF9 00 FF
0001FFFA 00 FF
0001FFFB 13 6E
0001FFFC 00 75
0001FFFD 00 57
0002E000 01 00
0002E001 FF 00
0002E002 FF 00
0002E003 FF 00
The second column IS confirmed to be UniPCemu's data in memory. The third column is Bochs' dump.
BTW dword at 0x2E000 is currently used as scratch memory for arith-logic tests 0xEE. Memory is accessed as DS:[0], where DS=0x14, with base=0x2E000
Just ran the testsuite on Bochs again. Ran the continue command to let it run until the permanent HLT. Then used Ctrl-C to break into the debugger. It's not a problem with UniPCemu there! Bochs errors out at the processor-specific arithmetic logic tests! It's at 0010:0000C002(the error jumped location) when it's processing those. So Bochs is having a problem with those tests(the E0 tests)!
00001382369e[CPU0 ] write_virtual_checks(): no write access to seg
00001382421e[CPU0 ] read_virtual_checks(): read beyond limit
00001382471e[CPU0 ] write_virtual_checks(): write beyond limit, r/w
00001382521e[CPU0 ] read_virtual_checks(): read beyond limit
00001382571e[CPU0 ] write_virtual_checks(): write beyond limit, r/w
00001385741i[CPU0 ] WARNING: HLT instruction with IF=0!
64799855992i[ ] Ctrl-C detected in signal handler.
Next at t=64813965152
(0) [0x0000000fc002] 0010:000000000000c002 (unk. ctxt): jmp .-4 (0x000fc000) ; ebfc
<bochs:2>
Edit: Having disabled those tests using the flag, I now see Bochs dumping all EE test results into it's debugger window.
The second column IS confirmed to be UniPCemu's data in memory. The third column is Bochs' dump.
Are you absolutely 100% sure? Because if I compare your post with a Bochs dump I've done, the 2nd middle column seems exactly like Bochs' data. For example the correct value for dword 2E000 is 0xFFFFFF01.
Well, I know for sure that the dumps of UniPCemu's Paging table locations contain those values in the dump and memory variable (0x27 and 0x67). So the second column is the UniPCemu memory and the third column is the Bochs memory. The first column is the address, the second column is UniPCemu, the third column is the crashing Bochs output.
So Bochs is having a problem with those tests(the E0 tests)!
Yes, 0xE0 are the undefined behaviours tests. Bochs is not a faithful replica of the i80386. Disable TEST_UNDEF and rerun the tests. Thanks for pointing out that test E0h should be disabled before comparing memory dumps. I'll add a note in the readme.
This is my new dump of the comparison of UniPCemu and Bochs after having disabled said test:
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFBF 56 46
0001FFE4 08 00
0001FFF0 08 00
Only a few bugs left! :D
OK.
First it writes value F0 to address FFFF at F000:FFF0. Also, FFFC=F5. Also FFFD=FF.
Thinking about that, it's the internal BIOS of UniPCemu performing it's startup routine(the CPU actually starts before the internal BIOS is handled, performing a CPU escaped instruction to make the emulator start it's yellow text boot, as well as providing internal interrupt handling(for when using the internal BIOS)).
So I'll need to not only reset the CPU(which already happens when starting the test386.asm ROM), but also clear any memory present(as it's been temporarily been modified to let the emulator's ROM run it's startup routine).
Said routine also loads the ROMs into memory, then resumes execution while resetting the CPU.
That explains the first half of said data. Luckily an easy to fix problem. Edit: Should be fixed now. Applied a full emulator reset before the emulator has loads the ROMs into memory. Then, the CPU reset afterwards has been removed, as the emulator reset has already performed this.
Having fixed said CPU reset to become an emulator reset, the first 64KB of errors disappears.
0001FFBF 56 46
0001FFE4 08 00
0001FFF0 08 00
Those are still left.
Added a little log of accesses to that location. The final writes(during the EE tests) show them being written:
00:04:17:35.05040: RealMMU 0001FFBF=56 at 0010:00007206
00:04:17:35.05088: RealMMU 0001FFF0=98 at 0010:00009859
00:04:17:35.05104: RealMMU 0001FFE4=08 at 0010:00007206
00:04:17:35.05440: RealMMU 0001FFF0=08 at 0010:0000985E