MXAO filtering causes artifacts

[rigopoui]

trafficstars

When setting the filtering property to either 1 or 2, it causes these line artifacts to appear. See images below, first without filtering, and then with filtering. They are especially visible on the door in the middle of the images. And in actual gameplay, are pretty visible in light-colored areas.

[martymcmodding]

This looks to me like a very corrupted depth buffer precision. If you are using logarithmic depth buffer, disable it. It barely does anything and only causes problems for AO altogether.

[rigopoui]

Logarithmic depth buffer setting is off. Far plane and multiplier are at default values. I am attatching a shot of the depth buffer as well. It looks ok to me. This example is from metroid prime on the Dolphin emulator, but I have run into this in other games. WoW 3.3.5 client comes to mind as another recent example.

[martymcmodding]

You can't tell from the debug view, because the depth buffer value needs higher precision.

What resolution do you play the game at and what ReShade version do you use?

[rigopoui]

Did some testing on another game, to make sure it's not a fluke of the emulator running in bizzare internal resolutions and such.

• Game: Hellblade - Senua's Sacrifice
• Resolution: Native 2560x1440 - no upscaling used
• Reshade version: latest - 6.3.1

It makes sense that depth precision is at fault as you say, as I see it happening on distant objects, where the precision is probably not enough to accurately judge their distance. Generally, when setting the fadeout distance to <0.5, the artifact is much less apparent. But still, I think a few months ago, I never saw this artifact, and I regularly use mxao in my games.

With filtering disabled:

With filtering set to '1':

With filtering set to '2': Shot of how it looks ingame:

Depth buffer view using the 'Display Depth' reshade effect:

[martymcmodding]

Late reply but alas: I've tested with a bunch of the games you mentioned but couldn't reproduce this anywhere which is... very strange to say the least. This is definitely not an easy bug to overlook and I don't understand why same ReShade + same game is not deterministic.

For sanity sake, can I see the global preprocessor definitions you're using?

I might also add, I'm working on a rather large update to all the depth-depending shaders that is intended to massively improve DLSS/FSR/TAA compatibility. Since this axes the current filtering entirely, quite possibly this bug is fixed along the way, even if you're not using these.

[rigopoui]

In the case of Senua, it was a fresh reshade install, with fully default pre-processor definitions. I've now tested in other games, and for me it seems to occur in every game, albeit to varying degrees.

Could I provide a log or something that would help you?

Perhaps it is somehow hardware specific? I'm running an AMD card, the 7900 GRE.

[gabriele2000]

7900 GRE

It could be the problem but the key is: drivers

I'm reading that this card has various problems... there's a bug that limits the overclocking but that will be fixed by a new driver, if it hasn't been fixed between November 2024 and today.

There are also reports about graphics glitches in various games, again driver's fault.

In my experience, "ReShade + specific game" is not something that's perfectly deterministic, since you have to count the hardware involved in the compiling, since every shader has to be compiled for different hardware.

IDEA: I believe you could gather additional info to help Marty debug your issue. By left clicking on the shader, there's an option called "show compiled result".

You could copy-paste one of those, maybe you can find the compiled result of the depth buffer alone, maybe using IMMERSE Launchpad's results

Example:

[rigopoui]

Long time no see 😂 Would any of these be useful?

I noticed an F__WriteDepthFeaturePS entry, here are the contents if it could be of use:

//
// Generated by Microsoft (R) HLSL Shader Compiler 10.1
//
//
// Resource Bindings:
//
// Name                                 Type  Format         Dim      HLSL Bind  Count
// ------------------------------ ---------- ------- ----------- -------------- ------
// __s0                              sampler      NA          NA             s0      1 
// __V__DepthInput_t                 texture  float4          2d             t0      1 
//
//
//
// Input signature:
//
// Name                 Index   Mask Register SysValue  Format   Used
// -------------------- ----- ------ -------- -------- ------- ------
// SV_POSITION              0   xyzw        0      POS   float       
// TEXCOORD                 0   xy          1     NONE   float   xy  
//
//
// Output signature:
//
// Name                 Index   Mask Register SysValue  Format   Used
// -------------------- ----- ------ -------- -------- ------- ------
// SV_TARGET                0   xy          0   TARGET   float   xy  
//
ps_5_0
dcl_globalFlags refactoringAllowed
dcl_sampler s0, mode_default
dcl_resource_texture2d (float,float,float,float) t0
dcl_input_ps linear v1.xy
dcl_output o0.xy
dcl_temps 1
sample_l_indexable(texture2d)(float,float,float,float) r0.x, v1.xyxx, t0.xyzw, s0, l(0.000000)
add r0.x, -r0.x, l(1.000000)
mad r0.y, -r0.x, l(999.000000), l(1000.000000)
div r0.x, r0.x, r0.y
mov_sat o0.xy, r0.xxxx
ret 
// Approximately 6 instruction slots used

I think you're right @gabriele2000 in suspecting driver problems. I don't think this occurred on my previous setup. I'll test some game on my old nvidia laptop to see if anything similar shows up. On the 7900 GRE, I'm on the latest driver and as of now this still occurs.

However, as of the the latest iMMERSE update, if I use the launchpad normals + the new _MARTYSMODS_TAAU_SCALE pre-processor this problem does go away.

It does hog quite a bit of extra performance this way for no reason other than fixing this problem, since I mostly use MXAO on older games that don't have any jittering on the depth buffer.

[martymcmodding]

The compiled result will unfortunately not help because that's the results from the DirectX compiler. This step is identical on all hardware and if there are decide differences, they happen after that, in the driver.

There are even stages in the Nvidia driver that detect certain instruction patterns and replace them with their own instructions, so sometimes this can cause trouble.

I was wondering: what far plane setting do you use (global preprocessor definitions, the big number)?

Also, try enabling the TAAU compatibility mode because that replaced the filtering. The current one is a bit ghosty but alright. You can do that by adding

_MARTYSMODS_TAAU_SCALE

to the preprocessor definitions with either a numerical value like 1.0 (for fullres TAA) or DLSS_QUALITY etc if you're running DLSS quality. There are macros for all the upscalers, see mmx_global.fxh.

[rigopoui]

@martymcmodding depth in most modern games is usually good with the default far plane and multiplier. In any case, even in older games/emulators where I have to do some tweaking to get a good depth buffer, or modern ones where it's good with the default values, I see the same issue regardless of the values I set to the depth preprocessor definitions.

The taau mode does indeed solve it. It is pretty ghosty without launchpad yeah. With launchpad it's fine, it's just more demanding than only mxao.

At this point I think it's safe to say that this is an AMD driver issue.

Edit: Could not reproduce this on my older nvidia-based system

[martymcmodding]

That is very insightful, thanks a lot. Also for the report on the other system. I've occasionally had weird behavior on AMD before but never got to the bottom of it.

My thoughts, albeit unverifiable:

• it looks characteristically like numerical instability, e.g. float16 vs float32. The SSAO seems unaffected though, against what I'd expect
• so it must be numerical instability inside the filter. The inputs can't be at fault, as the SSAO gets the same and has no issues. So it must be the instructions of the filter that exhibit vendor specific behavior
• code compiles to identical bytecode on all vendors so it must be in a later stage
• vendors have specific hardware instructions that merge regular opcodes, like when you do max(max(a,b),c) it becomes a max3 on hardware. I suspect my code is triggering something that changes the order of operations a bit and causes severe numerical instability. The linear guided filter I use is severely sensitive to that but I thought I tuned it not to be.

I'd still like to get to the bottom of this. Do you have Discord? If so, please join my server (see website) and message me. I feel iffy not knowing what causes this or how to avoid it, and I might have some ideas how to bypass this.