pcsx2 icon indicating copy to clipboard operation
pcsx2 copied to clipboard

PCSX2 Auto Test Suite

Open gregory38 opened this issue 9 years ago • 88 comments

A nice guy create a test suite: a https://github.com/unknownbrackets/ps2autotests

I create a nice script to run test (elf) tests/run_test.pl

Here the remaining item to do

  • [x] add an help of the run_test.pl script

  • [ ] fix bad test (priority are the crash/timeout)

  • [x] add multiple configuration on the runner

  • [x] find a way to blacklist bad test until there are fixed

  • [ ] implement more test

  • [x] add iop test on the runner

    ~~find a way to run the test suite on buildbot (linux)~~

So who do what ? @ramapcsx2 @ssakash @refractionpcsx2 @turtleli

gregory38 avatar Jan 18 '16 19:01 gregory38

i think @ramapcsx2 has already got some of these to run and highlighted some issues that need looking in to, he has a VM that can compile them and tested them on PCSX2 vs a TEST console.

refractionpcsx2 avatar Jan 18 '16 19:01 refractionpcsx2

actually I provided him a basic test runner so it used to run test in cygwin. The new test runner requires perl (I think it can work on cygwin too) but linux might be easier. (I mean for the PCSX2 part).

However you might be interested to add some tests. Which make me remember that cotton wrote a vu testsuite too??

gregory38 avatar Jan 18 '16 19:01 gregory38

I don't recall a test suite, i think some tests were done for testing VU oddities like multiple MUL's and ADD's in a row and getting the original answer or something ;p

refractionpcsx2 avatar Jan 18 '16 19:01 refractionpcsx2

Is it possible to get/check all cotton post with an attachment on the forum?

gregory38 avatar Jan 18 '16 19:01 gregory38

not for me, i dont have database access, that will be something for Bosit to do, if he's available.

refractionpcsx2 avatar Jan 18 '16 20:01 refractionpcsx2

@bositman we search a dev post of cotton with some attachment to run VU tests. If you have any hint.

Edit: or am I drinking too much !

gregory38 avatar Jan 18 '16 20:01 gregory38

@refractionpcsx2 Info on sysctrl test: Issue 1: infinte while loop in VifUnpackSSE_Dynarec::CompileRoutine

due to cl == wl == 0
so the else branch is always taken

Issue 2: Division by 0 because wl is 0

    if (!isFill) {
        // Accounting for skipping mode: Subtract the last skip cycle, since the skipped part of the run
        // shouldn't count as wrapped data.  Otherwise, a trailing skip can cause the emu to drop back
        // to the interpreter. -- Refraction (test with MGS3)
        uint skipSize  = (cl - wl) * 16;
        uint blocks    = v.block.num / wl;
        length += (blocks-1) * skipSize;
    }

gregory38 avatar Jan 18 '16 22:01 gregory38

If wl is 0 then the unpack won't work anyway, in filling write there has to be a WL, else nothing will be written. I guess we could bottle out of it completely, but that should never be the case.

I think there's a hack in unpack actually for a similar scenario that Kingdom Hearts does and that stops it doing stupid things.

Edit: yeh here it is

if ((vifXRegs.cycle.wl == 0) && (vifXRegs.cycle.wl < vifXRegs.cycle.cl)) { //DevCon.WriteLn("Vif%d CL %d, WL %d Mode %x Mask %x Num %x", idx, vifXRegs.cycle.cl, vifXRegs.cycle.wl, vifXRegs.mode, vifXRegs.mask, (vifXRegs.code >> 16) & 0xff); vifX.cmd = 0; return; // Skipping write and 0 write-cycles, so do nothing! }

I guess that could be modified to just say if wl == 0

refractionpcsx2 avatar Jan 18 '16 22:01 refractionpcsx2

Well don't know, but current code doesn't do what the PS2 does ;)

The others VIF issue is the programmation of the 2 bits mode. 3 is equivalent to 0. And I saw some basic mask.

gregory38 avatar Jan 18 '16 22:01 gregory38

Yeh but the PS2 probably doesn't have any code doing divides to work stuff out ;) If games do anything that stupid they deserve to crash lol. I can fix it if you really want me to though, but a game that does it would be useful.

what are you referring to 2 bits but 3 is 0? that rings a bell but what you said didn't make any sense xD

refractionpcsx2 avatar Jan 18 '16 22:01 refractionpcsx2

I miss you on IRC ;) We have a test that does it but I don't know if game does it (but you know). Edit: actually do we have game that freeze?

If you program the value 3 in the mode register the behavior of VIF will be the same as if you program the value 0.

gregory38 avatar Jan 18 '16 22:01 gregory38

oooh the mode register! yes, 3 is ignored, that should be the case in the current code, unless cotton fluffed it up :P I'll look in to it though.

refractionpcsx2 avatar Jan 18 '16 22:01 refractionpcsx2

Current behavior seems to be the offset mode (so 1).

gregory38 avatar Jan 18 '16 22:01 gregory38

virtual bool IsUnmaskedOp() const{ return !doMode && !doMask; }

I bet that's why, he's assumed if the mode isn't 0 it uses mode 1 and there is 1 line to do the special handling of mode 2, but nothing for 3 (that i can see)

refractionpcsx2 avatar Jan 18 '16 22:01 refractionpcsx2

We could put handling for this in to VifUnpackSSE_Dynarec::VifUnpackSSE_Dynarec(const nVifStruct& vif_, const nVifBlock& vifBlock_)

thats in newVif_Dynarec.cpp

what do you say? its either that or we put it after this line of the same file line 195

doMode = (upkNum == 0xf) ? 0 : doMode; // V4_5 has no mode feature.

refractionpcsx2 avatar Jan 18 '16 22:01 refractionpcsx2

I already quickly tried the line 195 but it isn't enough. Vif code is black magic for me.

gregory38 avatar Jan 18 '16 22:01 gregory38

what did you do? i presume you did

if(doMode == 3) doMode = 0; ?

if not then you should have xD

refractionpcsx2 avatar Jan 18 '16 23:01 refractionpcsx2

I assume the buildbot stuff is for me ;)

turtleli avatar Jan 19 '16 00:01 turtleli

Here the test diff with your patch.

-----------------------------------------------------------------------
/home/gregory/playstation/emulateur/pcsx2_merge/bin/PCSX2 --elf /home/gregory/project/ps2autotests/tests/dma/vif/mode.elf --cfgpath=/home/gregory/project/ps2autotests/tests/dma/vif/mode_cfg

--- /home/gregory/project/ps2autotests/tests/dma/vif/mode.PCSX2.out 2016-01-19 09:34:40.719208662 +0100
+++ /home/gregory/project/ps2autotests/tests/dma/vif/mode.expected  2015-12-22 13:51:51.142960920 +0100
@@ -46,13 +46,13 @@
   vumem(10): 00000004 - 00000005 - 00000006 - 00000007
   vumem(20): 00000008 - 00000009 - 0000000a - 0000000b
   vumem(30): 0000000c - 0000000d - 0000000e - 0000000f
-  row: 00001000 - 00001000 - 00001000 - 00001000
+  row: 0000000c - 0000000d - 0000000e - 0000000f

 mode 3 (mask diagonal):
   vumem(00): 00001000 - 00000001 - 00000002 - 00000003
-  vumem(10): 00000004 - 00001000 - 00000006 - 00000007
-  vumem(20): 00000008 - 00000009 - 00001000 - 0000000b
-  vumem(30): 0000000c - 0000000d - 0000000e - 00001000
-  row: 00001000 - 00001000 - 00001000 - 00001000
+  vumem(10): 00000004 - 00000001 - 00000006 - 00000007
+  vumem(20): 00000008 - 00000009 - 00000006 - 0000000b
+  vumem(30): 0000000c - 0000000d - 0000000e - 0000000b
+  row: 0000000c - 0000000d - 0000000e - 0000000b

 -- TEST END
-----------------------------------------------------------------------
I assume the buildbot stuff is for me ;)

Yes :+1: Honestly I don't know if we can do it but it will be interesting to do a summary of the issue so they can fixed one day. Note test requires a default config. The plan was to put something a default ini dir on the tests directory.

Command example (with only pass test)

./tests/run_test.pl --suite ~/project/ps2autotests/tests/cpu/ee_simd  --exe ./bin/PCSX2 --cfg bin/inis --cpu 2

gregory38 avatar Jan 19 '16 08:01 gregory38

I don't understand that diff? even the row's are different..

refractionpcsx2 avatar Jan 19 '16 09:01 refractionpcsx2

Oh sorry. The line starting with - are the from PCSX2 and the + are the expected. You can get the test and the expected here https://github.com/unknownbrackets/ps2autotests (test is stmod). It might be easier to understand what going on. (the while loop issue is stcycl test).

I think it would be easier for you to be able to run the test on your side. I think you can directly run the .elf on windows. @ramapcsx2 normally Perl could work on windows. I think it would be easier for everyone if we manage to get the script working on windows rather than relying on VM.

gregory38 avatar Jan 19 '16 10:01 gregory38

ok ill give it a go tomorrow, not sure ill have time tonight (i'm not home until late tonight)

refractionpcsx2 avatar Jan 19 '16 10:01 refractionpcsx2

no worry we have plenty of time :)

gregory38 avatar Jan 19 '16 10:01 gregory38

hmm just looking at the original "expected" file, mode 3 isn't seen as mode 0 at all, it's close but the row data gets overwritten.

every test sets the row to 0x1000 for each vector, however in the mode 3 tests the last set of vectors written to the VU overwrites the row contents (from the looks of things) it would be interesting to see how that unfolds with an extended set of data, if it's only ever the last written VU data that gets written back or if its the last of the first block (ok less likely, but stranger things have happened)

refractionpcsx2 avatar Jan 19 '16 10:01 refractionpcsx2

Ah. I better understand why the fix doesn't work :p (time to flee now ;) )

Feel free to extend the test. It would be nice if we can have an industrial test solution. It might help to fix corner case, but it will help a lots in case of code update and regression.

gregory38 avatar Jan 19 '16 11:01 gregory38

Well I would like to implement my unpack tests properly, I have got ELF's from when i did it before in a very "manual" method :P but would be good to actually use the SDK lol

refractionpcsx2 avatar Jan 19 '16 12:01 refractionpcsx2

Yeah, we still have many old tests, many with c code available. I'll collect them and give them to @gregory38 :) Getting the tester to run on Windows would certainly be preferable. I'll look into that.

ramapcsx2 avatar Jan 19 '16 12:01 ramapcsx2

Hum, I might need to port system call to perl equivalent. Otherwise I'm afraid it won't work but it must be doable (annoying but doable).

gregory38 avatar Jan 19 '16 12:01 gregory38

I need to learn how to use my build environment that rama gave me so I can actually build things properly :P I have got a cygwin setup with the open SDK on it (probably an archaic version now) which I used previously, but never actually used the SDK, I did every thing with arrays and register writes lol

refractionpcsx2 avatar Jan 19 '16 12:01 refractionpcsx2

@refractionpcsx2: It's embarrassingly simple with the vm, simpler than the old crappy msys way we used to do ;) Often you don't have to recompile the tests though, simple use the provided elfs. @gregory38 what am I doing wrong? https://i.imgur.com/1y1TdeA.jpg

ramapcsx2 avatar Jan 19 '16 12:01 ramapcsx2