SSR improvements

[Mannns]

As I explained in this thread : https://forum.babylonjs.com/t/ssr-when-i-rotated-a-mesh-the-reflection-shows-wrong-and-connected-to-the-mesh/30915/17, I worked on a new version of SSR.

I changed the main algorithm (intersection research) : I'm using 2D ray marching. I also changed some things:

• I corrected the final shading (little problem around the Fresnel formula that impacts the rendering of reflection)
• I use linear space reflectivity
• I let the opportunity to the user to apply a cubeTexture (Skybox or Probe) as a fallback
• I use reflectivity as a vec3 to allow reflection rendering with metallic-roughness workflow

I also defined a "backwardCompatibility" flag to avoid regression. This flag allows the user to use the first version of the algorithm instead of mine.

[deltakosh]

This is massive! thanks a ton

Would you be ok to also update the doc page?

[deltakosh]

cc @julien-moreau also for review

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[Mannns]

This is massive! thanks a ton

Would you be ok to also update the doc page?

Yes I will do it

[azure-pipelines[bot]]

Snapshot stored with reference name: refs/pull/12728/merge

Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/12728/merge https://gui.babylonjs.com/?snapshot=refs/pull/12728/merge https://nme.babylonjs.com/?snapshot=refs/pull/12728/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge#BCU1XR#0

[julien-moreau]

That's awesome @Mannns !!!! LGTM I'm just getting the branch to test it :)

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines[bot]]

Snapshot stored with reference name: refs/pull/12728/merge

Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/12728/merge https://gui.babylonjs.com/?snapshot=refs/pull/12728/merge https://nme.babylonjs.com/?snapshot=refs/pull/12728/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge#BCU1XR#0

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines[bot]]

Snapshot stored with reference name: refs/pull/12728/merge

Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/12728/merge https://gui.babylonjs.com/?snapshot=refs/pull/12728/merge https://nme.babylonjs.com/?snapshot=refs/pull/12728/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge#BCU1XR#0

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines[bot]]

Snapshot stored with reference name: refs/pull/12728/merge

Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/12728/merge https://gui.babylonjs.com/?snapshot=refs/pull/12728/merge https://nme.babylonjs.com/?snapshot=refs/pull/12728/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge#BCU1XR#0

[sebavan]

This is an amazing addition !!! @Mannns I ll do a final review once all @Popov72 comments have been resolved, I Love the idea of a SSR2 implementation not breaking back compat. Enjoy your vacations !!!

[Popov72]

Here's what I could come up with using the code there:

As you can see, there are a lot less artefacts (only at the base of the ball reflections) and they could be hidden with some blur. I'm going to play a bit more with the code and will make it available in case you are interested in integrating some parts.

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines[bot]]

Snapshot stored with reference name: refs/pull/12728/merge

Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/12728/merge https://gui.babylonjs.com/?snapshot=refs/pull/12728/merge https://nme.babylonjs.com/?snapshot=refs/pull/12728/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge#BCU1XR#0

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines[bot]]

Snapshot stored with reference name: refs/pull/12728/merge

Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/12728/merge https://gui.babylonjs.com/?snapshot=refs/pull/12728/merge https://nme.babylonjs.com/?snapshot=refs/pull/12728/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge#BCU1XR#0

[azure-pipelines[bot]]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s). To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[azure-pipelines[bot]]

Snapshot stored with reference name: refs/pull/12728/merge

Test environment: https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html

To test a playground add it to the URL, for example:

https://babylonsnapshots.z22.web.core.windows.net/refs/pull/12728/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge https://sandbox.babylonjs.com/?snapshot=refs/pull/12728/merge https://gui.babylonjs.com/?snapshot=refs/pull/12728/merge https://nme.babylonjs.com/?snapshot=refs/pull/12728/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/12728/merge#BCU1XR#0

[Popov72]

Another follow up:

Without the backface depth buffer	With the backface depth buffer

Look at the reflection of the balls on the floor and the reflection of the red ball in the blue ball. These screenshots have been taken with step=30.

The backface depth buffer is used to compute a per-pixel geometry thickness instead of relying on a constant value (see https://github.com/kode80/kode80SSR).

Another one (step=10):

Without the backface depth buffer:

With the backface depth buffer:

Source code, for reference: https://github.com/BabylonJS/Babylon.js/compare/master...Popov72:Babylon.js:ssr-improve

[Mannns]

Another follow up: Without the backface depth buffer With the backface depth buffer

Look at the reflection of the balls on the floor and the reflection of the red ball in the blue ball. These screenshots have been taken with step=30.

The backface depth buffer is used to compute a per-pixel geometry thickness instead of relying on a constant value (see https://github.com/kode80/kode80SSR).

Another one (step=10):

Without the backface depth buffer:

With the backface depth buffer:

Source code, for reference: [master...Popov72:Babylon.js:ssr-improve](https://github.com/BabylonJS/Babylon.js/compare/master...Popov72:Babylon.js:ssr-improve

Love the rendering ! I will have a look ASAP Just wonder how it behaves with a scene with walls or high mesh density.. ?

[Popov72]

Just wonder how it behaves with a scene with walls or high mesh density.. ?

It has all the problems of SSR, so depending on the scene it can fail miserably or be quite good.

For eg, here are 3 screenshots of the HillValley scene where I have simply enabled the new SSR (no tweaking of the meshes/materials):

I'm going to try to add a blur pass, as implemented here.

[Popov72]

Blurring added.

No blur:

Some blur (roughness=0.1):

Lot of blur (roughness=1):

[Mannns]

Blurring added.

What you did is awesome !!

[Popov72]

The PG I'm using for my testings: https://playground.babylonjs.com/#PIZ1GK#639

Note that it works only with the geometry renderer at the time and not with the prepass renderer yet.

[Mannns]

The PG I'm using for my testings: https://playground.babylonjs.com/#PIZ1GK#639

Note that it works only with the geometry renderer at the time and not with the prepass renderer yet.

May I help you ? I added the depth in the screenSpaceReflectionConfiguration file. Then I observed that the textureSampler is weird when enabling prepass

[Popov72]

Yes, the prepass renderer configuration was missing.

When fixed, the problem with the display was because the MRT is not created right from the start by the prepass renderer and the call to ScreenSpaceReflection3PostProcess._getTextureSize() was failing.

I have fixed the problem: http://localhost:1338/#PIZ1GK#642

However, we can now see that there are some default roughness on the road, something we don't see with the geometry renderer => the prepass renderer is right on this one, it's the geometry renderer that does not write the correct roughness in the gbuffer texture when using the standard material.

Maybe you can have a look and see if you can fix the problem?

There's also another problem when enabling the blur pass (ssr.blurDispersionStrength > 0) with the prepass renderer, it seems we can't retrieve the input texture of a post process with postProcess.inputTexture, it returns undefined...

[sebavan]

I converted as a draft as I enjoy a lot the progress but want to prevent any early merge during the finalization. You guyz are awesome !!!

[Mannns]

However, we can now see that there are some default roughness on the road, something we don't see with the geometry renderer => the prepass renderer is right on this one, it's the geometry renderer that does not write the correct roughness in the gbuffer texture when using the standard material.

Maybe you can have a look and see if you can fix the problem?

I just fixed it.

In geometry.fragment.fx it's: reflectivity = texture2D(reflectivitySampler, vReflectivityUV);

instead of: reflectivity.rbg = texture2D(reflectivitySampler, vReflectivityUV).rbg

[Popov72]

Good catch, I have also fixed it in my sources and a couple of other ones related to linear/gamma space for the reflectivity.

I still have some problems when comparing the geometry renderer / prepass renderer:

• the prepass renderer does not handle transparent materials correctly when writing to the normal/reflectivity render target textures. Blending is enabled whereas it should not, but I don't know if it's possible to enable blending on some RT and not on some others, as we still need blending for the color RT... I tried to look for this issue in Google but could not find anything relevant to WebGL2 (in OpenGL you have glDisablei to disable blending on a specifc RT)... @sebavan Do you know if that is possible? I think I saw someone speaking of an extension for WebGL2 that would let use different alpha blending parameters on a per render target basis (so it would probably also allow to enable/disable alpha blending on a per render target basis) but could not google it...
• I have sometimes some differences in the reflections and it seems it's the geometry renderer which is at fault, I need to dig more...

[Popov72]

I have sometimes some differences in the reflections and it seems it's the geometry renderer which is at fault, I need to dig more...

It is also linked to the blending mode in the MRT. It appears it is sometimes better with the blending enabled when writing the depth/normal in the prepass renderer mode and sometimes it is worse, depending on the camera view...

[yuqianma]

Great result ! Any plan to merge it?

[Popov72]

It still needs some more work (PBR is not ok), I should be able to work on this again in the next weeks.