Temporal Antialiasing (TAA)

[JMS55]

Objective

• Implement an alternative antialias technique
• TAA scales based off of view resolution, not geometry complexity
• TAA filters textures, firefly pixels, and other aliasing not covered by MSAA
• TAA additionally will reduce noise / increase quality in future stochastic rendering techniques
• Closes https://github.com/bevyengine/bevy/issues/3663

Solution

• Add a temporal jitter component
• Add a motion vector prepass
• Add a TemporalAntialias component and plugin
• Combine existing MSAA and FXAA examples and add TAA

Future Work

• Prepass motion vector support for skinned meshes
• Mip biasing for sharper textures, and or unjitter texture UVs https://github.com/bevyengine/bevy/issues/7323
• Compute shader
• More extensive tests/examples with moving transparency/alpha masking
• Investigate performance/accuracy of Rg16Float vs Rg32Float for motion vector texture

Maybe Future Work

• Additional flicker reduction (more permissive clipping when confident in history)
• More disocclusion techniques such as stencil, depth, velocity, luminance (configurable)
• Possible faster motion vector prepass by skipping meshes marked as non-moving (camera motion vector only)
  • Often need prepass fragment shader for normals anyways, but maybe do normals + motion vectors then switch pipelines and do just normals

---

Changelog

• Added MotionVectorPrepass and TemporalJitter
• Added TemporalAntialiasPlugin, TemporalAntialiasBundle, and TemporalAntialiasSettings

[JMS55]

I can't figure out the doc CI failure related to [OrthographicProjection]. Earlier on in the file another doc comment uses the exact same link, and it works. I don't know why it's failing on TemporalJitter - I checked, I didn't make a typo or anything.

[JMS55]

It was pointed out to me that skinned meshes are probably broken for the velocity prepass. I will need to fix that.

[JMS55]

@robtfm applied your feedback. Also added a "reset button" to TAA, which is both useful in general for the user, but also prevents that decrease in brightness for the first few frames due to blending with the initially empty history texture (black).

Still need to figure out the velocity for the background / ghosting against the background, and skinning support. We may want to push skinning support to a future PR though, I don't have a good plan to deal with it yet.

Ghosting against the background can be solved with a fullscreen initialization pass to set the correct velocity for the background, but that feels like a bad solution to me.

[JMS55]

Do you know if the tonemapping or anywhere else where you're adding 0.000001 might raise the black level from black to dark gray?

I tried it out. TAA on vs off with a black clear color has no difference noticeable to the human eye.

[Elabajaba]

Do you know if the tonemapping or anywhere else where you're adding 0.000001 might raise the black level from black to dark gray?

I tried it out. TAA on vs off with a black clear color has no difference noticeable to the human eye.

I went and tested it with a colour picker, and TAA doesn't cause raised blacks. (also, unless you're checking on an OLED you probably wouldn't be able to tell by just looking at it anyways)

edit: For clarity, if you disable ambient light then you get 0,0,0 as expected with a solid black material.

[JMS55]

@robtfm the flicker reduction stuff isn't perfect, but it improves some specular and SSAO noise when standing still. There's still quite a lot of flicker for instance on the back of the flight helmet in the scene viewer. Solving that without ghosting will be much harder, and won't be solved in this PR. It's more of an anti-noise than anti-flicker.

[bakcxoj]

What else needs to be done for this to be merged?

[JMS55]

What else needs to be done for this to be merged?

It needs review and approval from SME-Rendering. Code wise, it's pretty much done for now. The only major missing piece is support for skinned meshes, but I'm going to leave that for a future PR.

[github-actions[bot]]

Example alien_cake_addict failed to run, please try running it locally and check the result.

[github-actions[bot]]

Example alien_cake_addict failed to run, please try running it locally and check the result.

[JMS55]

Current issues:

• Running the shader_prepass example gives me an error thread 'Compute Task Pool (0)' panicked at 'No prepass fragment shader provided for shader_prepass::CustomMaterial.', C:\Users\Jasmine\Projects\bevy\crates\bevy_pbr\src\prepass\mod.rs:397:171
• The following systems should maybe be a part of PrepassPipelinePlugin, instead of PrepassPlugin?

.add_system(
    apply_system_buffers
        .in_set(RenderSet::Prepare)
        .in_set(PrepassLightsViewFlush)
        .after(prepare_lights),
)
.add_system(
    prepare_previous_view_projection_uniforms
        .in_set(RenderSet::Prepare)
        .after(PrepassLightsViewFlush),
)

• Unsure about jitter range, we need to double check the math

[robtfm]

prepare_previous_view_projection_uniforms

pretty sure that won't be needed for the pipeline plugin (which is just for shadows) without the actual prepass plugin as well

[JMS55]

pretty sure that won't be needed for the pipeline plugin (which is just for shadows) without the actual prepass plugin as well

It's actually needed due to the shadow pass reusing the binding group, which contains that uniform, but I don't think it's necessary to add that system - I guess the empty uniform buffer still works.

[JMS55]

@robtfm as I suspected, there's a bug with the shadow pass and PreviousViewProjectionUniforms. If you add the following to the lighting.rs example

.add_plugins(DefaultPlugins.set(PbrPlugin {
    prepass_enabled: false,
    ..default()
}))

Then it crashes due to missing PreviousViewProjectionUniforms resource. If you init the resource in PrepassPipelinePlugin instead of PrepassPlugin, then it crashes in SetPrepassViewBindGroup::render(), as it can't find PreviousViewProjectionUniformOffset...

EDIT: Trying to fix this...

[JMS55]

Ok, fixed that. Seems to work, but please double check the previous commit and ensure I didn't introduce new bugs haha.