FSR3 Quality Issues in Unreal Engine PlugIn

[BleuBleu]

Im opening a second issue for the UE5 plugin here since I do not know if this quality regression is linked to #133 or if its an issue with the Unreal Integration itself.

Problem: In UE5, the upscaled image quality of FSR3 is unstable, seems to converge much less than with FSR2. I have experienced this in our own game (The Outlast Trials - Red Barrels) and decided to try it in stock UE5 just to be 100% sure it wasn't an issue with my integration in our (unsupported) UE 4.27.

Here are some GIFs of a simple scene showing the issue. GIFs are not ideal, its much easier to see in action.

FSR2:

FSR3:

Observations:

• Both struggle with some high-frequency shadows in the middle and some high-frequency checkerboard in the distance, lets ignore that.
• FSR3 in unable to converge for the grid lines on the floor, they remain pixelated/fuzzy where FSR2 is able to lock onto them.
• FSR3 is unable to resolve the thin parts of the fence (right side) whereas FSR2 is able to lock onto them
• FSR3 struggles with the metallic shading on the fence in the middle, FSR2 is able to converge to a stable image.
• Overall there is a constant shimmering on almost every edge with FSR3 as if the algorithm is unable to converge, this does not happen with FSR2. A bit hard to see on the GIF ill admit, but absolutely there in person.

Now lets look at something more challenging, such as hair.

FSR2:

FSR3:

Observations:

• FSR3 completely fails here. I tried toggling the hair de-dither, etc. It just changes the thickness of the hair, the flickering is just as obvious.
• FSR3 show slight shimmering on the gridlines on the floor, as if the algorithm is struggling to lock on to them.

Repro steps:

1. Install a fresh UE 5.5
2. Install latest FSR3 (3.1.3)
3. Install latest FSR2 (2.2.1) and fix a few compilation issues since the last supported UE was 5.3
4. Enable one of them at a time
5. Set quality to Ultra Performance (3x), disable reactive mask, turn off sharpening. Other settings do it too, its just easier to see in 3x and the reactive mask is just to set an even playing field, my scene has no transparency anyway.
6. Create a small map with some geometry, some alpha tested stuff too.

Until these issues are resolved, we are not planning on offering FSR3 to our players.

-Mat