Animation Improvements

[sebavan]

Collecting here the list of improvements related to the animation system we would like to see in our version 7:

• [x] Animation Blending
• [x] Animation Group Blending/Layering
  • Layering is not yet in Babylon.js, but is currently being tested on the avatar prototype.
• [x] Animation Sequencing
• [x] Skeleton Retargeting (identical skeleton shape and identical bone hierarchy)
• [ ] Skeleton Retargeting (different skeleton shape and identical bone hierarchy)
• [ ] Skeleton Retargeting (different skeleton shape and different bone hierarchy)
• [x] Bones management helper (position in world space....)
• [x] Animation management helper (visual debugger without meshes....)
• [x] Can we remove Skeleton.needInitialSkinMatrix and everything related to it? It seems it's an old feature not used anymore... => can't be removed, see https://github.com/BabylonJS/Babylon.js/pull/14063
• [ ] Skeleton viewer: show bone axis
• [ ] Skeleton viewer: show leaf bone correctly => if bone.length, use that, if not, compute a "suitable" length...

The issues will be refined in their own one once will start the next release.

Here is a list of related forum thread/issues this should be able to address:

• https://forum.babylonjs.com/t/animations-storing-key-frames-in-typed-arrays/36566/18
• https://forum.babylonjs.com/t/how-to-bind-separate-glb-file-such-as-cloth-etc-to-armature/37574
• https://forum.babylonjs.com/t/usd-3000-reward-anyone-figured-out-retargeting-yet/36629/30
• https://forum.babylonjs.com/t/additive-animations-distorted/37751
• https://forum.babylonjs.com/t/usd-1000-reward-moving-bones-to-different-position/37929
• https://forum.babylonjs.com/t/gltf-animation-loops-when-i-click-how-to-stop-the-loop/38477/6
• https://forum.babylonjs.com/t/baked-texture-animations-with-animation-groups/32518/11
• https://forum.babylonjs.com/t/skeletal-mesh-performance/40522

[kzhsw]

Some other features might worth considering: Interleaved animation data: https://github.com/KhronosGroup/glTF/issues/1864 EXT_animation_quantization: https://github.com/KhronosGroup/glTF/pull/1991 Resampling animation: https://github.com/donmccurdy/glTF-Transform/blob/main/packages/functions/src/resample.ts

[FlorentMasson]

What about supporting blending with Baked Texture Animations?

[kzhsw]

Where is the memory optimization now? Is it permanently out of the plan?

[Popov72]

Memory optimization is a big job! It's not out of the plan, but it's not on the list of short-term features.

[Popov72]

What about supporting blending with Baked Texture Animations?

I don't see how this can be done except by moving the entire animation system (or at least a significant part of it) to the GPU...

[punkoffice]

Pardon my noobness, so has this been integrated into Babylon version 7.1.0? Does this mean when I load an animation with ImportAnimations and set the target node to the bone of some avatar then retargeting will now work without twisted bones?

[Popov72]

Retargeting is one of the sub-task of #14963