Dual quaternion skinning

[failip]

Related issue: #20324

Description

Dual quaternion skinning is a widely-used alternative to linear blend skinning. However, it's currently not available in three.js. In this draft, I present a proof-of-concept implementation of dual quaternion skinning. Unlike traditional skinning techniques that use transformation matrices, dual quaternion skinning uses dual quaternions. To apply this technique, the bone transformation matrices must be converted to their corresponding dual quaternion representation. I've implemented this conversion step in the vertex shader for simplicity, but it may be more efficient to perform the calculation on the CPU.

Example Quaternion skinning on the left, linear on the right

Problems

• ~~Models using the dual quaternion skinning seem to be translated twice. I could not figure out why this was happening. Maybe someone can spot the mistake.~~
• ~~More complex models seem to sometimes rotate in the wrong direction causing artifacts.~~

TODO

• ~~Apply the new skinning approach to the object normals~~

References

• https://github.com/sketchpunklabs/ossos/blob/main/examples/threejs/_lib/SkinDQMaterial.js
• http://rodolphe-vaillant.fr/entry/29/dual-quaternions-skinning-tutorial-and-c-codes
• https://users.cs.utah.edu/~ladislav/kavan07skinning/kavan07skinning.pdf
• https://github.com/toji/gl-matrix/blob/master/src/quat.js
• Example Model

[github-actions[bot]]

📦 Bundle size

Full ESM build, minified and gzipped.

Filesize dev	Filesize PR	Diff
670.3 kB (166.2 kB)	673.1 kB (166.9 kB)	+2.78 kB

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

Filesize dev	Filesize PR	Diff
450.6 kB (108.9 kB)	452.4 kB (109.3 kB)	+1.84 kB

[failip]

I moved the matrix to dual quaternion conversion CPU side. The Skeleton class turns the offset matrix into a dual quaternion and a scale vector saves this data in the Bone Texture. This theoretically frees up 4 bytes per bone, but I'm currently just adding those four bytes as padding to keep consistent with the size of the old texture. Dual Quaternion Skinning now has to be enabled on both the SkinnedMesh for the Shader and on the Skeleton for the Bone Texture which is a bit clunky because they should never be used independently.

child.useDualQuaternionSkinning = true;
child.skeleton.useDualQuaternion = true;

Is there a nicer way of implementing this?

[failip]

Currently the depth buffer/shadows are broken if both a normally skinned mesh and a dual quaternion skinned mesh are used as can be seen here. This is caused by both SkinnedMeshes using the same DepthShader, and the Shader then sampling the dual quaternion data as matrix data. Is there a way of forcing both Objects to use a different depth shader?

[mrdoob]

Are there any downsides to simply replacing the current code with dual quaternions?

I would prefer to not have two different approaches.

[brainexcerpts]

Are there any downsides to simply replacing the current code with dual quaternions?

Just my opinion from someone that have some experience about skinning in the industry:

If you mean by that replacing the standard linear blending skinning (LBS) by dual quaternions skinning (DQS) I don't think it's a good idea as it would break compatibility with models designed for LBS (the vast majority).

The problem being, skinned mesh are to be designed with one or the other technique in mind. Something skinned for LBS will give really poor deformation results when run with DQS; that's because the skin weights really react differently in both approaches.

Although it's usually easier to quickly produce a skinned mesh with DQS, DQS itself has some downsides that makes it unsuitable for certain models. Ultimately they are complimentary, rather than DQS being a complete replacement of LBS. So there is no "one size fits all" for mesh deformation IMO.

Broadening this conversation, you can see that advanced engines like Unreal Engines are now developing fully customizable pipelines (a.k.a Deformer Graph in UE) to allow anyone to define which compute shader should be ran on which mesh. But maybe that's a bit out of Three.js' scope.

My take is that, while a lot of people would be really happy to have DQS in Three.js their would be even more people disappointed if LBS disappeared, forcing them to tweak all their models' skin weights specifically for Three.js/DQS.