bevy icon indicating copy to clipboard operation
bevy copied to clipboard

Load morph target buffers from GLTF assets

Open james7132 opened this issue 3 years ago • 3 comments

Objective

GLTF supports storing morph target data in meshes. bevy should be able to load these buffers properly.

Solution

  • Add a Vec of MorphTargets to store the per-target displacements.
  • Extend bevy_gltf to load the associated displacements into these buffers.
  • Encode all morph targets as a 2D texture to support sampling more than 8 morph targets at once.

Followup

A PR for rendering the displacements should be created. This will involve the following:

  • Bind the texture and sampler.
  • Add a shader def for specializing mesh shaders for morph target support.
  • Include a uniform that includes the weights and indexes of the highest weighted morph targets along with a length of how many active morph targets there are.

Open Questions

  • What's a reasonable balance between performance and artistic flexibility? Ideally up to 256 active morph targets should be supported for maximum flexibility, but that would require 2KB of weights/indexes in every mesh uniform.
  • This implementation is limited to the max dimension of a texture divided by 3.Can this be avoided by using 2D texture arrays?
  • Will this break any future efforts to do mesh batching?
  • Morph target displacements are typically very small, and almost all 0. Can we save RAM and VRAM without significant loss of quality by using TextureFormat::Rgba16Float?

james7132 avatar Jan 19 '22 08:01 james7132

How will morph targets and skeleton animation work together? If I'm not mistaken, usually skeleton animation is applied after the vertex positions are moved by morph targets. How are we going to achieve this wit Bevy?

kodra-dev avatar Feb 17 '23 11:02 kodra-dev

They're composable. We'll likely need to apply morph target displacements before skinning. If we move to compute shader based skinning, morph target animation can likely just be another compute shader step in the pipeline that deforms the mesh.

james7132 avatar Feb 17 '23 14:02 james7132

For keyword searches: this feature is also known as blend shapes (Unity, Maya) or Shape Keys (blender)

nicopap avatar Mar 07 '23 12:03 nicopap

I've adopted this feature. Making progress on the morph-target branch of my bevy fork.

The idea is to use a 3D (array) texture, where each pixel is a single component of morph-target animated attribute.

nicopap avatar Mar 17 '23 10:03 nicopap

Closing in favor of #8158.

james7132 avatar Mar 22 '23 15:03 james7132