Displacement Effect

[defektu]

Fixes https://github.com/playcanvas/engine/issues/5554

@mvaligursky requested PR from #5554 to look into it.

Known issues with PR is in issues thread.

It might be a poor implementation in terms of engine's workflow.

LitShader-forward-proc & LitShader-ShadowPass_1_0-proc does not get #define DISPLACEMENT argument possibly because of my poor implementation.

Project file in #5554 works perfectly (probably due to shader chunk rebuilds when replacing a chunk)

Did not implemented checks for implementation cause of draft PR

I confirm I have read the contributing guidelines and signed the Contributor License Agreement.

[mvaligursky]

Would you have any way to use this? Engine example or Editor project?

[defektu]

Scene for changing transformVS and engine example with use_local_engine

[mvaligursky]

I had a quick look, and I think:

1. you should use heightMap for this instead of introducing new texture channel. The useDisplacement would then use this map as displacement instead of for parallax.
2. I think you need something like this to have normals attribute added to the vertex shader:

Give these a go please and let me know if there are further issues - there might be with supplying UVs to the vertex shader too, but maybe not with the heightMap.

[mvaligursky]

Hi @defektu - any thoughts / progress on this? It'd be great to finish off if possible, thanks.

[defektu]

Sorry @mvaligursky, nowadays are a bit rough workwise.

Using heights was possible but uv's and texture is not passing through to shadowpass. (vertex's are displacing but shadow pass won't in this case.)

Shadows are rendered through Standart material shader as far as I inspected. Build process won't take the litOption arguments (like texture_shadowMap) while shadows are processing (useHeights=null). Or I might doing something wrong.

I can explore any thoughts in my spare time.

[defektu]

Today I noticed standart.js has // all other passes require only opacity part. So ShadowPass is not getting any info about anything except opacity.

[mvaligursky]

Yep, we'll need to adjust this to pass on the height if used as well.

[defektu]

Also vertex_normals needed. But it does not pass into shadow pass as semantics as well Vertex shader attribute "vertex_normal" is not mapped to a semantic in shader definition, shader [Shader Id 2 LitShader-ShadowPass_1_0-proc]

[defektu]

Shaderpass does not get any litOptions (turns out to be undefined) getShaderVariant passes this.shaderOptBuilder.updateMinRef if it is not FORWARD_PASS. So do you have any suggestions about this topic. Besides shadowPass it is working as intended (displacing the surface).

[mvaligursky]

I suspect the StandardMaterialOptionsBuilder will need to be updated to generate required options in the shadow pass - probably in _updateSharedOptions, which I think is shared between forward and shadow pass, but not sure now based on you seeing litOptions being undefined.

Perhaps update the PR with what you have, using the height texture, and I'll see how I can fix the shadows using your example.

[defektu]

Rolled back previous changes and added mesh-displacement example draft.

[defektu]

Somehow I got it working with shared options. I will cleanup everything and send PR.

[defektu]

Pushed working version with minimalOptions = false. It is working as intended like this.

[defektu]

Effect fully works for now but noticed env goes blank (weird frustrum culling behaviour causes that) Can't state if it is caused by PR or something else.

[mvaligursky]

I fixed the culling that few days ago .. you're just probably not up to date. Is it all ready for the review you reckon? Ping me when it is, I'll check it out, thanks!

[defektu]

Yes it is up and running @mvaligursky. UV transformations are not implemented. displacementOffset is nice to have and can be implemented by introducing new material attribute.

[defektu]

Pushed new commit for displacementOffset too.

[defektu]

Hi @mvaligursky, It's all ready to review.

Rebased branch with latest updates I accidentally force repushed everything again sorry about that. Culling is fixed with updates from origin. heightMap still affects when displacement is used, this causes some kind of weird artifacts with big displacements. I am suggesting seperating heightMapFactor and adding displacementScale.

Seperating displacementScale will give possibility to remove useDisplacement boolean. We can disable effect if displacementScale == 0. That might be better for UI cleanness with single slider to do everyting at once.

[mvaligursky]

I am suggesting seperating heightMapFactor and adding displacementScale.

Great idea! Perhaps we should call it just displacement, default it to 0?

[willeastcott]

It seems several map types support some kind of factor or intensity map/value (e.g. normal, emissive). Can we consider some common naming for that?

[mvaligursky]

Yeah I was looking at it as well .. and saw

heightMapFactor
bumpiness
occludeSpecularIntensity
specularityFactorTint
metalness
iridescenceThickness

and didn't see any easy common naming we could easily move to.

[defektu]

There is specularityFactor. Since effect uses heightMap, displacementFactor and displacementOffset is good?

[mvaligursky]

There is specularityFactor. Since effect uses heightMap, displacementFactor and displacementOffset is good?

yeah actually, I missed you added an offset too. Sounds good in that case to me.

[defektu]

Hey @mvaligursky, Updated PR & ready for review.

Decided to keep useDisplacement attribute for code readability. It sets based on displacementFactor. Cleaned up mesh-displacement example.

I guess it is good to keep useDisplacement attribute in material parameters for debugging?

Not a code dev mainly so ignore commit chains <3

[defektu]

Added draft docs for attributes.

Things to consider:

• This effect uses UV0 and R channel of height map.
• You can't offset or transform map yet.
• If there is no heightMap, useDisplacement will break vertex chunk.

[willeastcott]

Hey @mvaligursky and @slimbuck - can we sync on this PR? I'm keen to make a decision on what to do with it.

[Maksims]

Is there a way to change heightMap channel for displacement? Also will it apply tiling and offsets?

Another question, will it be viable/common when devs use displacement in combination with parallax? In such case the heightMap textures might be different or the same?

[defektu]

Hi @Maksims, I think this vertex shader needs some special attributes for offsetting and tiling (which is totally possible). Concurrent options are bound to fragment shader. Don't think that back propagation is not possible unless changing where uniforms defined.

Not really experienced with engine, so this needs crosscheck.

[defektu]

Back from service. Ready to do changes if needed.

[mvaligursky]

This should be a lot easier (very easy) to do using a ShaderMaterial for custom shaders - which would support skinned and instanced meshes too, see more info here: https://github.com/playcanvas/engine/issues/6835

Path to add it to the StandardMaterial is similar as before, but little simplified. But to add it to it I think we should ideally wait for https://github.com/KhronosGroup/glTF/issues/948 to shake out (could be a while!), to add exactly what is needed.

If you're keen to try and write an engine example using ShaderMaterial, that'd be a welcome addition. As that would likely be a separate PR, I'll close this one, but very much appreciate your time and efforts here @defektu .