xemu nv2a: Fix pixel center offset problem

This addresses the difference in pixel center between OpenGL and the DirectX 8 approach used by the hardware.

Note that the resultant behavior still does not fully match the hardware, where screen-space pixel coordinates below (x + 0.5625) => x, but it is close enough to fix issues with surface-target rendering. Test HW results With this patch, xemu will produce correct output for the render-target but will not fully match the geometry-based test output.

Fixes #725 Fixes #57

Feb 15 '22 18:02 abaire

This appears to resolve shadow issues in 5451001a, but only at 1x scaling. On 2x or above, the behavior is the same as master. bfbb fix_pixel_rounding demo

Debated between posting here or in the issue, but this seems to be related to the code change so I opted to put it here.

Feb 23 '22 00:02 antangelo

This removes the edge bloom (top and left) present in Fable: The Lost Chapters. :+1:

Mar 19 '22 01:03 dracc

Progression: artifacts on borders have disappeared for splinter cell double agent and chaos theory with this PR on native scaling:

Mar 19 '22 10:03 Fabxx

lights

Splinter cell 1 has been fixed, no more greenish bar on the right side

Mar 19 '22 22:03 Fabxx

Still need to fix this at higher render scales, and I just found that it doesn't quite work correctly on M1 (at 0.5 there is already a red bar along the left side, though not along the top) so there may be some device/GL dependency to work through as well.

Apr 25 '22 06:04 abaire

Also fixes Counter Strike Fix

May 01 '22 20:05 LeRutY

The current state of this PR also causes some corruption in certain games like Splinter Cell:Double Agent when combined with #879 (which unmasks the issue by allowing compounding errors).

In the pgraph test for #879, I see that there's a yellow column along the right hand side of the texture after it is drawn the second time. I suspect that this happens in games as well, leading textures that go through multiple phases of compositing to be smeared (this is exactly what @revix-0 has reported in SC:DA w/ this fix).

Here is a screenshot from xemu, Screenshot_20220505_070346

and this is the same screenshot zoomed in to look at the right column Screenshot_20220505_071114

Removing this PR and running #879 alone results in correct output.

May 05 '22 15:05 abaire

It looks like there is differing behavior between cases where we are rendering to the framebuffer versus to a texture.

When rendering direct to the framebuffer, it appears to be correct that we should adjust the coordinate by flooring the (screenspace) output of the xbox vertex shader + 0.5625 to match hardware rounding behavior.

When rendering to a texture, it seems like even without this PR (and with any offset values I've tried) there is texture smearing, which makes me think that it's related to the way xemu handles render targets (it looks like it's being filtered rather than faithfully copied).

May 06 '22 23:05 abaire

Here is an illustration of the problem. This screen is the result of compositing the left hand side of a texture on top of itself (leaving the right hand side as the initial value, a hand-defined checkerboard pattern that I've verified is correct by dumping the memory passed to glTexImage2D in pgraph_upload_surface_data).

Whenever xemu does compositing, it does a partial pixel shift, despite the s2t_rndr being set up with correct coordinates and even if I force GL_NEAREST filtering. I'm not yet sure where the failure is coming from, but this is probably the root cause of the texture corruption that can be seen in combination with #879.

Screenshot_20220506_211642

May 07 '22 04:05 abaire

A note on debugging this: Unfortunately renderdoc seems to fail to capture interim states for some of the textures. If I take the hand-initialized texture and just render it to the standard color surface I can look at the input texture in renderdoc and confirm that it is as expected (a crisp checkerboard).

If I add a self-modifying step (input texture renders to a surface pointing at the same memory address), renderdoc ends up showing the final modification as the input stage, so it becomes very confusing as to whether the input texture was corrupted somehow (doesn't seem likely given the above check) or if renderdoc ends up updating its view of the texture after xemu updates it post-render. This step also corrupts the input (in renderdoc) for the test draw that is not self modifying it.

E.g.,

Create texture manually
Draw texture to normal color buffer. If you stop here and run in renderdoc, the input texture will be exactly as expected
Draw texture to a surface at the same address as the texture (on HW this would literally render into itself). If you examine the input texture for this phase in renderdoc, it'll show the final (corrupted) output. If you examine the input texture for step 2 in renderdoc, it'll also show the corrupted output, indicating a problem with renderdoc's resolution (it seems to only correctly track the final state).

May 07 '22 16:05 abaire

With nsight it's a bit more clear:

When creating the initial texture from the surface, the result is correct (no smearing).
When doing the first draw (rendering into the surface target) the smearing is introduced.

So I think we can rule out the s2t_rndr, but I'm still left wondering why we don't get consistently shifted results (e.g., if I don't render back into the surface at the same hwaddr as the texture, I get a pixel-aligned image).

May 07 '22 17:05 abaire

It looks like the way xemu handles the WINDOW_CLIP commands may be incorrect, which also contributes to this problem (and may explain uninitialized data along the outer edges of surfaces)

May 09 '22 02:05 abaire

WINDOW_CLIP differences do not explain the behavior.

I have a fix that gets a pixel perfect match in my test case, but still breaks SC:DA (it does a bunch of downscaling + jitter to do lighting effects and with this fix the resultant overlay is misaligned compared to the plain framebuffer).

May 10 '22 01:05 abaire

can you rebase this with master please so i can do more testing

May 18 '22 20:05 ryzendew

@RyzenDew - it should've already been automatically mergable, but rebased anyway

May 18 '22 21:05 abaire

fable is looking good but npc's weird outline is still a bit off just glad the white bits are gone

May 18 '22 22:05 ryzendew

Note: The current state of this PR breaks the highlight texture generation in NFS MW (#890) where the low res version of the color buffer appears to suffer from a pixel offset issue. Applying this PR causes the output to be completely black.

May 30 '22 15:05 abaire

Fixes Deus Ex Invisible War