Implement vertex shading

[ywmaa]

Implements Vertex Shading solves : https://github.com/godotengine/godot/issues/43093

• [x] OpenGL
• [x] Vulkan Mobile
• [x] Vulkan Forward+

Bugs :

• [x] ~Vertex Shading receives shadows from one light source in GLES3~ ~~No Shadows will be casted in Vertex Lighting~~
• [x] When using Forward+, vertex lighting doesn't render unless you get very close to the light node.
• [x] When using lightmaps, DirectionalLights with the Static bake mode still affect lightmapped surfaces.
• [x] This PR completely removes shadows in the compatibility renderer regardless of whether vertex shading is enabled
• [x] The "force vertex shading" project setting should prompt users to restart the engine
• [x] The "force vertex shading" project setting appears to only work for StandardMaterials (it should work regardless of the material type and should apply to all materials in the scene)
• [x] Use simplified light model for vertex shading like 3.x
• [x] Directional Light does nothing if shadows is enabled both in Vertex Lighting or Pixel Lighting (GLES3 only Issue)
• [x] Shadows for vertex shading in Vulkan/Forward Renderers.
• [x] In Forward+, the vertex lighting will pop in on the edges of the viewport. (Disable Light clustering with vertex lighting)
• [x] When you enable shadows in Compatibility, the light is drawn twice: one with per-pixel shading, one with per-vertex shading
• [x] when adding a varying in gdshader, it errors out

~~Note: This PR implements Vertex-Lighting without shadows.~~ Note: This PR implements Vertex-Lighting with Pixel-Shadows. Production edit: Pixel shadows are only available for the first DirectionalLight3D in the scene. Omni and spot lights cannot cast shadows on vertex-shaded materials (like in Godot 3.x), except in the Compatibility rendering method where lights with shadows are rendered with a multipass approach.

[ywmaa]

For some reason, only the material preview shows the vertex shading, in the viewport it looks like it doesn't use vertex shading. Any Idea why ?

Per-Pixel Per-Vertex

[AnalogFeelings]

For some reason, only the material preview shows the vertex shading, in the viewport it looks like it doesn't use vertex shading. Any Idea why ?

Per-Pixel Per-Vertex

That does not look like vertex shading at the bottom material preview

[ywmaa]

are you sure this is not vertex-lighting ?

this is normal pixel lighting :

[ywmaa]

mmm I guess I did vertex shadow not vertex lighting, fixing it.

Edit : lol, it is actually the opposite, shadow calculation should be added in vertex.

[ywmaa]

I guess I need an advice from the Rendering Team, currently I am almost duplicating functions for the vertex-shading, should I just use Preprocessor to move every thing to vertex shading, or should I keep duplicating the stuff, because we may remove features from the vertex shading ?

[ywmaa]

Progress Vertex lighting (Lighting + Shadows) in GLES3/Compatibilty renderer:

currently known bug : Vertex Shading receives shadows from one light source in GLES3, and if you enable shadow casting on the second light, it won't cast light at all to the mesh.

Bug images :

Second light no shadows :

Second light with shadows :

Edit :

Well I just tried Godot 3.5, and vertex_lighting doesn't cast shadows at all

was it intended ?

[Calinou]

Regarding why the second light with shadows disappear, remember that Godot 4's GLES3 performs multipass lighting if you use multiple lights with shadows. This is different from Godot 3 GLES3 (and more comparable to Godot 3 GLES2, although it used multipass for non-shadowing lights too).

Well I just tried Godot 3.5, and vertex_lighting doesn't cast shadows at all

Yes, real-time shadows can't be received by vertex-lit surfaces. Only baked lightmap shadows can be seen on them.

In GLES3 (unlike GLES2), it should be possible to sample the shadowmap texture in vertex() if you want real-time shadows to be visible on vertex-lit surfaces, but it's not trivial. The shadow detail will also be limited by the receiving object's mesh complexity. (If this behavior needs to be disabled, the Do Not Receive Shadows material flag can be used.)

[ywmaa]

In GLES3 (unlike GLES2), it should be possible to sample the shadowmap texture in vertex() if you want real-time shadows to be visible on vertex-lit surfaces, but it's not trivial. The shadow detail will also be limited by the receiving object's mesh complexity.

so do you think I should remove shadows from vertex-lighting ?

[Calinou]

so do you think I should remove shadows from vertex-lighting ?

I think you can remove them for now. Support for receiving shadows in vertex-lit materials can be implemented in a future PR (if it's feasible at all) :slightly_smiling_face:

[ywmaa]

Now Compatibilty/GLES3/OpenGL Rendering should be ready for Vertex-Lighting testing and regression testing.

I will continue work on Vulkan Mobile + Vulkan Forward+

[ywmaa]

SOOOOO Vertex Shading now works in Vulkan

Unfortunately I had to do Mobile + Forward in one commit, as both depend on scene_forward_lights_inc.glsl

Now TIME for review, but remember :

[ywmaa]

My comment on shadows for vertex lighting :

I think it is possible to cast shadows using vertex_shading.

But I think vertex shading is mainly useful for performance reasons, and probably realtime shadows are costly.

So it doesn't make a lot of sense to have realtime shadows in vertex lighting.

I may implement shadows in another PR if there is a demand for it.

Personally I don't need shadows in my game for vertex lighting.

[ywmaa]

@Calinou these two commits should fix your issues (except flicker in Vulkan Mobile, or maybe it is fixed too ?) I tested locally, but I need a confirm.

[AnalogFeelings]

https://github.com/godotengine/godot/assets/51166756/a30135c2-2046-4275-8538-63d094844d90

It seems some polygons become unlit if they become partially out of view?

EDIT: I am on Forward+ by the way.

[ywmaa]

@AnalogFeelings Yes this happens to me too, I think this is light clustering in Forward+ and not a bug.

Maybe calinou knows.

[AnalogFeelings]

I also noticed setting the viewport to lighting only causes HEAVY artifacts.

https://github.com/godotengine/godot/assets/51166756/447b0a30-0a96-4a22-8d1e-d1b60553ade3

[ywmaa]

I also noticed setting the viewport to lighting only causes HEAVY artifacts.

Interesting, I'll check it out.

[ywmaa]

@AnalogFeelings

I am unable to reproduce your issue :

But I noticed that in lighting mode it shows vertex lighting like pixel lighting, so that will probably need a fix.

[Calinou]

But I noticed that in lighting mode it shows vertex lighting like pixel lighting, so that will probably need a fix.

Lighting mode replaces all materials in the scene with a basic ShaderMaterial that uses per-pixel lighting (it might just be the fallback material). It can't know whether the original material used vertex shading or not. The Lighting debug draw mode's appearance will probably have to be toggled with a project setting.

[ywmaa]

Just fixed some bugs that I noticed from my work.

[ywmaa]

But I noticed that in lighting mode it shows vertex lighting like pixel lighting, so that will probably need a fix.

Lighting mode replaces all materials in the scene with a basic ShaderMaterial that uses per-pixel lighting (it might just be the fallback material). It can't know whether the original material used vertex shading or not. The Lighting debug draw mode's appearance will probably have to be toggled with a project setting.

I was thinking do we really need to have vertex lighting debug view ? or pixel lighting debug view is fine ?

[ywmaa]

Just did a rebase to latest master commit, I don't understand why it got all of my commits again like they are totally new.

[akien-mga]

Just did a rebase to latest master commit, I don't understand why it got all of my commits again like they are totally new.

That's what rebasing does, it reapplies your commits on top of the latest master branch, so those commits are new as far as Git and GitHub are concerned (new hash, new committer date).

[TechMan06]

May I ask how do I use this on my project? I'm making a PS1 styled game and I really need a way to add vertex lighting to it, I spent a lot of time trying to find a solution to this bug as soon as I noticed it.

[AThousandShips]

You can click the code button in the top right and clone this branch, or wait for the code to get merged

[Parsnip]

Heyo, how does this look now? Rebase and early 4.3 merge hopefully. 🙏

[ywmaa]

Just did a rebase, I hope it still works, and I hope that I don't break anything xd.

[Parsnip]

Just did a rebase, I hope it still works, and I hope that I don't break anything xd.

For what it's worth I rebased and tested it on my end at the end of 4.2 beta cycle and everything seemed to work great. Friend of mine will be very happy when this is merged. 😎

[Calinou]

Tested locally (rebased on top of master f8a2a9193662b2e8c1d04d65e647399dee94f31e), it mostly works as expected.

I found 2 remaining issues as of cecc8e4:

• When using Forward+, lighting information for vertices close to the screen will disappear. This may be due to light clustering, as this doesn't happen when using Mobile or Compatibility.

https://github.com/godotengine/godot/assets/180032/7b124cf9-d1ff-4662-b66c-558a71887988

• Specular lobes from lights don't appear, unlike per-pixel shading. I'd expect them to appear like per-pixel shading, but calculated on every vertex (Gouraud shading).
  • You can test this by playing around with a light's Specular property. It should ideally work with all light types.

Visual comparison

Left: Per-pixel shading Right: Per-vertex shading

Forward+

Mobile

Compatibility

[ywmaa]

@Calinou Are we able to know if the performance of it is really good/better than pixel lighting ?

For some reason on my RTX 3070 mobile, I didn't see performance difference when I tried it in my level ?

Am I doing something wrong with the lighting, or high end GPUs have low cost with pixel-shading ?

I have only one directional light, so maybe it isn't costly at all.

Regarding your issues 1- forward clustering previously was broken and I fixed it, but the problem is that vertices near the camera end some become out of view, so basically the clustering removes/skips the lighting process on the vertex, should I do an offset or something ?

2- maybe I forgot to implement specular in vertex lighting, so I will check it out.