xemu
xemu copied to clipboard
xemu-project
nv2a: Adjust NaN handling to be similar to HW
Fixes #365 by forcing treatment of NaN values generated in the vertex shader to follow the same pattern as HW.
Unfortunately this does not address the fact that on HW, +/- NaN * 0 (or very close to 0) should be 0. This difference in behavior is also responsible for #1008.
This also does not address the same issue for +/- inf and the shader version currently used by xemu explicitly notes that operations on inf/NaN's are undefined (which may explain why Otogi textures are not blacked out on M1 mac).
Hey @NZJenkins I wasn't sure if you were already looking at a fix, but I started to suspect this might be partly to blame for #801 so I threw together a partial fix to unblock my debugging there.
This is mostly a proof of concept at the moment as it doesn't fully match the HW behavior and because some investigation will need to be done on the performance impact (and possibly some optimization)
glitch in NGB this PR
may relate to issue https://github.com/mborgerson/xemu/issues/783
same glitch in DOA2U
@ko81e24wy where is the nearest save point so I can reproduce that?
I think you just need to start a new game
Hey @NZJenkins I wasn't sure if you were already looking at a fix, but I started to suspect this might be partly to blame for #801 so I threw together a partial fix to unblock my debugging there.
No not looking at a fix. Did something similar for cxbxr where all outputs are de-NaN-ified, but probably need more tests to determine the correct behaviour.
For texcoord and fog outputs, additional processing happens on the VS output values before they are used in the PS, which may interpret NaN, INF, etc. differently to the mapping that happens when they are passed directly from the VS to the PS. Not sure how Xemu shaders are set up but that might need to be accounted for here.
Unfortunately this does not address the fact that on HW, +/- NaN * 0 (or very close to 0) should be 0.
The Min test is using "subnormal" float values- it might be interesting to try it with the minimum "normal" values instead, and see if they behave differently to 0. Apparently it's more complex to support subnormal numbers correctly in hardware, so for a GPU it's likely not implemented (added test cases - subnormals appear to be treated as 0 https://github.com/abaire/nxdk_pgraph_tests/pull/65)
@ko81e24wy where is the nearest save point so I can reproduce that?
I think you just need to start a new game
There was a stupid mistake in my code that is now fixed, can you test again when you have a chance?
I am just going to work and test with a combine version with this PR merged
that white glitch was gone,and I dont know if this PR relate to https://github.com/mborgerson/xemu/issues/783
@ko81e24wy cool, thanks for double checking!
I have not tried to debug 783 yet, so if this PR happens to fix the missing lighting that'd be a nice bonus. I'm not exactly sure what that screenshot should look like compared to HW though; are you saying it looks more correct with this PR than before?
@ko81e24wy cool, thanks for double checking!
I have not tried to debug 783 yet, so if this PR happens to fix the missing lighting that'd be a nice bonus. I'm not exactly sure what that screenshot should look like compared to HW though; are you saying it looks more correct with this PR than before?
that white issue remind me of that light effect,the last PR just as before with that effect missing.
Was hoping this would fix the black graphics of Inside Pitch 2003 but guess that is a separate issue from this pr since that game still looks like this:
Was hoping this would fix the black graphics of Inside Pitch 2003 but guess that is a separate issue from this pr since that game still looks like this:
@mysteria25 as far as I can see from the compatibility page there is no bug tracking this problem, please file one and it'll get looked into eventually (PRs are definitely not the place to report untracked issues). Grabbed by the Ghoulies has a black texture problem related to texture compression; there are many ways for things to fail so it's entirely possible what you're seeing with that game is something novel that will need a specific fix.
This doesn't entirely match HW behavior on my GTX 1070 but it is much closer and fixes #365 despite the remaining departure around infinite values and zeros.
current handling causes this bug where the character is highlighted in proximity of lights
current handling causes this bug where the character is highlighted in proximity of lights
Looks like limiting the NaN fix to fog will fix that regression and align with Matt's suggestion, will update and re-push in a bit.
I test the lastest build of this PR,and it crash when I run xemu. xemu.log
Bad merge, should be fixed now. Thanks for letting me know!
@abaire could you rebase this pr with the work you did on Match inv_w qualifier to attribute qualifier, Thanks
Can we get another pull request or rebase or however that works so there is a working download available for this branch for testing? The one from July is expired and no longer available.
