bevy
bevy copied to clipboard
Load morph target buffers from GLTF assets
Objective
GLTF supports storing morph target data in meshes. bevy should be able to load these buffers properly.
Solution
- Add a Vec of
MorphTarget
s to store the per-target displacements. - Extend bevy_gltf to load the associated displacements into these buffers.
- Encode all morph targets as a 2D texture to support sampling more than 8 morph targets at once.
Followup
A PR for rendering the displacements should be created. This will involve the following:
- Bind the texture and sampler.
- Add a shader def for specializing mesh shaders for morph target support.
- Include a uniform that includes the weights and indexes of the highest weighted morph targets along with a length of how many active morph targets there are.
Open Questions
- What's a reasonable balance between performance and artistic flexibility? Ideally up to 256 active morph targets should be supported for maximum flexibility, but that would require 2KB of weights/indexes in every mesh uniform.
- This implementation is limited to the max dimension of a texture divided by 3.Can this be avoided by using 2D texture arrays?
- Will this break any future efforts to do mesh batching?
- Morph target displacements are typically very small, and almost all 0. Can we save RAM and VRAM without significant loss of quality by using
TextureFormat::Rgba16Float
?
How will morph targets and skeleton animation work together? If I'm not mistaken, usually skeleton animation is applied after the vertex positions are moved by morph targets. How are we going to achieve this wit Bevy?
They're composable. We'll likely need to apply morph target displacements before skinning. If we move to compute shader based skinning, morph target animation can likely just be another compute shader step in the pipeline that deforms the mesh.
For keyword searches: this feature is also known as blend shapes (Unity, Maya) or Shape Keys (blender)
I've adopted this feature. Making progress on the morph-target
branch of my bevy fork.
The idea is to use a 3D (array) texture, where each pixel is a single component of morph-target animated attribute.
Closing in favor of #8158.