Supermodel icon indicating copy to clipboard operation
Supermodel copied to clipboard

Visual studio build quirk(s)

Open toxieainc opened this issue 1 year ago • 27 comments

Using my own VS2022/2019 builds, e.g. both Daytona 2 games and Scud Race feature a quirk where it seems the gas pedal (all pedals?) is stuck pressed at the beginning of the race, but only on certain tracks (e.g. on D2 medium/advanced course, SR expert course).

This does not happen with the official gcc based build.

toxieainc avatar Oct 03 '24 07:10 toxieainc

My guess is our friend the uninitialized variable. I fixed a whole bunch but there are more.

dukeeeey avatar Oct 03 '24 07:10 dukeeeey

Maybe it's worth trying a clean build? Try compiling with Rebuild Solution instead of Build Solution and see if it changes anything.

Occasionally I'll get weird issues - usually Supermodel failing to start - that are fixed by making a clean build. Also I personally turn off link-time code generation (LTCG) since that seems to make corrupt builds less frequent.

gm-matthew avatar Oct 03 '24 21:10 gm-matthew

Yes, i already tried all kinds of things. Also changing various compiler options, and trying to init more class members (just to be sure), but so far the problem remains with the VS builds.

toxieainc avatar Oct 08 '24 08:10 toxieainc

I guess next step would be to run a real mem tracker like Dr.Memory on it to see where/if uninit'ed mem is used or if there is some other unknown glitch.

toxieainc avatar Oct 08 '24 08:10 toxieainc

So far no luck with uninit'ed mem. :/ The tool reports tons of issues in the Intel GFX driver though ;)

toxieainc avatar Oct 08 '24 18:10 toxieainc

Have you tried some good old fashioned printf() debugging to see where the input values appear to be corrupted? Starting to suspect a code generation issue and unfortunately, if it's somewhere in the CPU emulator, it'll be hard to find, but you'll be able to assess at least whether the input system is returning the expected values (which would indicate something is happening downstream in the CPU emulator). I haven't tried building with MSVC yet but this shouldn't be too hard to debug. I may be able to look into it after Monday. Does it also happen when the debugger is built? If so, the debugger will be immensely helpful. Input values are usually read once per frame in a single place.

trzy avatar Oct 10 '24 20:10 trzy

@trzy ReadInputs() does not show any activity for Steer,Acc or Break for the m_game.inputs & Game::INPUT_VEHICLE case. Or is there more to check?

toxieainc avatar Oct 16 '24 19:10 toxieainc

This sounds bad. But: does it show activity when compiled with gcc or in the stages where it works? If so (and I do think it should unless I am missing something obvious), then I think the problem is inside the input system.

On Oct 16, 2024, at 12:36 PM, toxie @.***> wrote:

@trzy https://github.com/trzy ReadInputs() does not show any activity for Steer,Acc or Break for the m_game.inputs & Game::INPUT_VEHICLE case. Or is there more to check?

— Reply to this email directly, view it on GitHub https://github.com/trzy/Supermodel/issues/203#issuecomment-2417775469, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJZ3RJNJP5CTVEC5VOGFS3Z3254BAVCNFSM6AAAAABPJE5WOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJXG43TKNBWHE. You are receiving this because you were mentioned.

trzy avatar Oct 16 '24 19:10 trzy

Misunderstanding. It works if i press keys, etc, but it does not show any activity if not pressing anything, so the problem that the car seems to have acceleration and brake pressed does not seem to come from the inputs.

toxieainc avatar Oct 16 '24 19:10 toxieainc

@trzy Any other hints what/how i could continue debugging?

toxieainc avatar Oct 17 '24 07:10 toxieainc

You’ll have to trace deeper into the actual game code. What I typically do is 1) compile with debugger enabled and 2) to find the program counter address where some register is accessed, I place a printf() statement that prints the value of ppc_get_pc() in the appropriate handler (in this case, the function in Model 3 that returns analog inputs). Unfortunately, it will then become a more painful process because you’ll have to figure out where these inputs ultimately get stored in the game’s RAM, where they are picked up by other parts of the game code, etc. I don’t think the debugger has memory watches (maybe it does?) but, depending on where in RAM they are stored, sometimes they are accessed using a consistent register addressing mode that is text-searchable in the game code. You can then disassemble the entire game (from the RAM dump using ppcd, which is buildable by setting a macro in PPCDisasm.cpp) and try to identify points where the steering inputs are read.

The problem is, who knows how deep the problem runs.

Just off the top of my head, one potential culprit is the floating point code. Can you play with MSVC optimization options, specifically around floating point math?

On Oct 17, 2024, at 12:22 AM, toxie @.***> wrote:

@trzy https://github.com/trzy Any other hints what/how i could continue debugging?

— Reply to this email directly, view it on GitHub https://github.com/trzy/Supermodel/issues/203#issuecomment-2418762270, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJZ3RM3OHT2XLTETU5HLGDZ35QULAVCNFSM6AAAAABPJE5WOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJYG43DEMRXGA. You are receiving this because you were mentioned.

trzy avatar Oct 17 '24 07:10 trzy

Okay, thanks! I fear i will not have enough spare time soon to really dig into this problem THAT deep. But i will try to fool around with compiler options.

toxieainc avatar Oct 17 '24 07:10 toxieainc

The more I think about it the more it seems likely that the culprit is something like FP math or some other weird instruction.There used to be a PowerPC emulator stress tester — I think part of the SheepShaver project. There might also be a regression tester in Dolphin we could borrow. Sent from my iPhoneOn Oct 17, 2024, at 12:48 AM, toxie @.***> wrote: Okay, thanks! I fear i will not have enough spare time soon to really dig into this problem THAT deep. But i will try to fool around with compiler options.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

trzy avatar Oct 17 '24 08:10 trzy

Okay, so basically in the PPC core then?

toxieainc avatar Oct 17 '24 08:10 toxieainc

That’s where I’d look first. Quick things to check also, as sanity checks, are that the final values returned from the relevant functions in Model3.cpp (the register handlers, at the final return statement) are as expected and not being corrupted there. Also the read handlers themselves, e.g. Read8(), contain a big switch statement. Can’t imagine any problem there but it’s one more place to do some sanity testing in. Besides that I think the most likely culprit is either the floating point code setting some PowerPC flag incorrectly, thereby causing a branch to go awry, or some other weird condition. I wouldn’t over index on float stuff just because I suggested it but it’s one place to start where I think we had seen trouble before (albeit across all compilers because the code was just wrong).In this case, it really does smell like a code generation or optimization issue if gcc and clang don’t have a problem.Sent from my iPhoneOn Oct 17, 2024, at 1:42 AM, toxie @.***> wrote: Okay, so basically in the PPC core then?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

trzy avatar Oct 17 '24 14:10 trzy

Is the gcc or clang path compiled with fast-math?

toxieainc avatar Oct 17 '24 20:10 toxieainc

Nope not that I can see. Rules.inc and Makefile.Win32 don’t have that. They just use -O3 and set C++17 mode.Sent from my iPhoneOn Oct 17, 2024, at 1:06 PM, toxie @.***> wrote: Is the gcc or clang path compiled with fast-math?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

trzy avatar Oct 17 '24 20:10 trzy

Have you tried a 32bit vs 64bit build?

dukeeeey avatar Oct 17 '24 20:10 dukeeeey

I really nailed it down to the math opts in PPC now. Looking into what is causing it now.

toxieainc avatar Oct 17 '24 20:10 toxieainc

How did you narrow it down?Sent from my iPhoneOn Oct 17, 2024, at 1:38 PM, toxie @.***> wrote: I really nailed it down to the math opts in PPC now. Looking into what is causing it now.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

trzy avatar Oct 17 '24 21:10 trzy

Switching compile options. And the ppc files are the only ones that 'reacted' to this (=causing the misbehavior). I still want to track down the exact place(s)/function(s) in there, so that hopefully we could just make things more explicit and in the future could enable more aggressive optimizations if needed.

toxieainc avatar Oct 18 '24 16:10 toxieainc

Im sure the rounding mode for the project got changed at some point which effects the powerpc float rounding

dukeeeey avatar Oct 18 '24 16:10 dukeeeey

Yes, i saw that. And in some sense this is also a bit risky. From my understanding so far, the rounding mode is changed, as soon as the game code calls that opcode. The problem is that this is done 'globally', so all other Supermodel code will also be affected by that then. Thats also something i want to look into.

toxieainc avatar Oct 18 '24 21:10 toxieainc

Oh my. Going forward, the only reliable solution seems to be to emulate all rounding modes 'by hand' for each PPC fp instruction. Only visual c++ seems to offer a reliable way to set the rounding mode, and while gcc and clang offer a compile option, in practice that one seems to be not fully reliable. :/

toxieainc avatar Oct 18 '24 23:10 toxieainc

Why does it work in gcc and clang then? And are we sure this error is due to rounding modes?

On Oct 18, 2024, at 4:09 PM, toxie @.***> wrote:

Oh my. Going forward, the only reliable solution seems to be to emulate all rounding modes 'by hand' for each PPC fp instruction. Only visual c++ seems to offer a reliable way to set the rounding mode, and while gcc and clang offer a compile option, in practice that one seems to be not fully reliable. :/

— Reply to this email directly, view it on GitHub https://github.com/trzy/Supermodel/issues/203#issuecomment-2423353167, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJZ3RPADM2YCNB5PNLEED3Z4GII5AVCNFSM6AAAAABPJE5WOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRTGM2TGMJWG4. You are receiving this because you were mentioned.

trzy avatar Oct 18 '24 23:10 trzy

std::fesetround

dukeeeey avatar Oct 18 '24 23:10 dukeeeey

Yes, its 100% due to the rounding modes. I will fix this for visual c++, as it provides a reliable way to support fesetround. But going forward, gcc and clang may just also show bad behavior at some point from what i read, as the behavior there is not well defined, even with -frounding-math enabled.

Plus, what i think is even worse, fesetround changes behavior not just for the PPC code parts, but globally for all fp operations in supermodel.

toxieainc avatar Oct 18 '24 23:10 toxieainc

I now fixed all the mentioned issues above in aboves PR.

The only caveat is that the GCC and clang behavior is still based on 'luck', same as before. So you guys should definetly enable -frounding-math for GCC and clang, but according to my net research the behavior even then is still not well defined. But all other solutions, like manually emulating the rounding modes, would be total overkill though and also not exactly trivial.

Sidenote: The FMA PPC opcodes may suffer from insufficient precision (as for the double variants its not actually fused). And also all NaN checks may be optimized away if one turns on aggressive math optimizations (which is not the case at the moment i think).

toxieainc avatar Oct 20 '24 14:10 toxieainc

Sidenote: The FMA PPC opcodes may suffer from insufficient precision (as for the double variants its not actually fused). And also all NaN checks may be optimized away if one turns on aggressive math optimizations (which is not the case at the moment i think).

I believe that the way nearly all of the floating point arithmetic instructions work on PowerPC is that they use an FMA unit with a 56x28 bit multiplier, where fmul sets FRB to 0.0 and fadd sets FRC to 1.0. Double precision multiplies require two cycles; one to multiply the higher bits of FRC and one to multiply the lower bits, and the two results are added as a part of the second FMA operation.

I have a PowerBook with a PowerPC 603 which I used to fix a rounding issue that was causing the car to go off track in the attract mode of Daytona 2 BOTE; I could use it to help make the floating point instructions even more accurate if so desired.

gm-matthew avatar Oct 20 '24 17:10 gm-matthew

Thanks for the feedback! I would assume that double precision isn't used much if these take double the cycles. Are the fp32 opcodes okay like-is?

toxieainc avatar Oct 20 '24 17:10 toxieainc