godot icon indicating copy to clipboard operation
godot copied to clipboard

Add RetargetModifier3D for realtime retarget to keep original rest

Open TokageItLab opened this issue 1 year ago • 3 comments

Closes https://github.com/godotengine/godot-proposals/issues/3379

RetargetModifier3D

Add RetargetModifier3D. Currently Godot performs retargeting by overwriting rest. However, that means discarding the bone rest which set in the DCC, and it is quite possible that the axis of rest will be in an unintended direction. It makes re-targeting useless for specifying IK pole vectors and setting constraints for specific axes.

This PR solves this problem by retargeting the parent skeleton with the rest defined in the SkeletonProfile and the child skeleton holding the original rest with the RetargetModifier that will be added in this PR. Also keep in mind that parenting is required to guarantee process order.

image

The Profile specified to RetargetModifier3D is used only as a list of bone names, so if you want to retarget only specific bones, you can create a custom SkeletonProfile with arbitrary bone names only and set it to the RetargetModifier3D.

It has the following two types of retargets by Use Global Pose option.

Use Global Pose option

If enabled, retargeting is enabled when the parent of the source bone is deformed, even if the target bone is less than the source bone. However, as a limitation, the lengths of the bones must be exactly equal. For example a specific use case, adding a dummy bone with zero length helps share animation between models with different numbers of bones.

https://github.com/user-attachments/assets/44a86290-4950-4d2b-9ebb-99036e7e0e85

If disabled, retargeting is not performed when the parent of the source bone is deformed if the target bone is less than the source bone. However, it is possible to retarget bones of different lengths without destroying the appearance of the bones. For example a specific use case, you can retarget a chibi character from a skeleton that matches your real body shape for XR motion tracking. In addition, position, rotation, and scale can be handled individually.

image

Post importer change

Add option to separate unmapped bone library

image

Add option to use RetargetModifier to keep original rest

image image

Note that the Rename Bone option must be enabled.

image

TokageItLab avatar Oct 04 '24 20:10 TokageItLab

Is there particular tests you want us to look at? I'll try to review this as soon as possible.

fire avatar Oct 05 '24 01:10 fire

I put the project that I used to test the behavior on my end. I would appreciate it if you could retarget other models or test the combination with XRModifier like one more RetargetModifier to the GeneralSkeleton parent to retarget from the real body shape skeleton with tracking. cc @Malcolmnixon

retarget-modifier-test.zip

Godette / CC by SirRichard94 Blender Chan! / CC by SearKitchen Alicia Solid / DWANGO Co., Ltd.

TokageItLab avatar Oct 05 '24 07:10 TokageItLab

It may be possible (not so at the current this PR) to modify it so that Skeleton can create a children list even if it is not a direct child of RetargetModifier like:

ParentSkeleton
└RetargetModifier
 └Node
  └ChildSkeleton <- detect

but it will need to be tested.

I will try to improve this in a later Follow up PR, as it could be applied to SkeletonModifier lists that are direct children of Skeleton to organize tree view like:

Skeleton
├Node
│├ModifierA_1
│└ModifierA_2
└Node_2
 ├ModifierB_1
 └ModifierB_2

TokageItLab avatar Oct 05 '24 08:10 TokageItLab

I have tested this on https://github.com/V-Sekai/godot-humanoid-project where we retarget a character body without a UpperChest to a character body that has does not have a UpperChest but only has a Chest.

Note that there a difference in animation in the godot-humanoid-project but it is unrelated to this pr.

fire avatar Oct 15 '24 16:10 fire

@TokageItLab Does this resolve the Add a bone expander (to be able to change the local scale) priority?

fire avatar Nov 21 '24 18:11 fire

@fire No, Bone expander should be superseded by the option to add a cancel bone by constrain as discussed in https://github.com/godotengine/godot-proposals/discussions/9885#discussioncomment-9665995; It adds a constrained bone without length that applies the reverse scale to the child of the bone you want to cancel the scale. We have discussed this with lyuma in the past and we agree.

This PR helps to implement Constraint (by keeping original rest), but is not directly related to it. Although it might be useful in terms of facilitating retargeting when adding a zero length bone.

Canceling non-uniform scales in local space requires allowing non-orthogonal scales in the bone pose, but non-orthogonal scales do not conform to the glTF specification and are incompatible with the TRS3D track, which prevents compression and hinders the implementation of streaming animation playback.

TokageItLab avatar Nov 21 '24 18:11 TokageItLab

Thanks!

Repiteo avatar Nov 26 '24 19:11 Repiteo