MotionVAEs-WebGL icon indicating copy to clipboard operation
MotionVAEs-WebGL copied to clipboard

possibility to use glb rigged Avatars

Open arpu opened this issue 4 years ago • 5 comments

Hey,

this is awesome work! do you think its possible to use a rigged SkinnedMesh Avatars Bones? like the Avatar are created on https://fullbody.readyplayer.me/ the result glb has the rigged bones

Bildschirmfoto von 2020-10-22 15-30-05

arpu avatar Oct 22 '20 13:10 arpu

I've never done rigging, so don't know what the process is. I think it should be possible. What do you need to rig the character model? Joint positions and orientations?

belinghy avatar Oct 22 '20 17:10 belinghy

i can create a small demo to show to set the bones position/rotation with THREEJS the model is rigged with the bones and this bones can be transferred by position/rotation

arpu avatar Oct 22 '20 21:10 arpu

i think this is a good example https://threejs.org/docs/#api/en/objects/SkinnedMesh

arpu avatar Oct 22 '20 21:10 arpu

and here a small not complete example with Inverse kinematic https://glitch.com/edit/#!/dashing-floor?path=cool-file.js%3A1%3A0

@belinghy is this the right way to look at this?

arpu avatar Oct 22 '20 21:10 arpu

I couldn't understand the code you linked, because I'm unfamiliar...

But from the three.js skinning and blending example, it seems the animation is driven by these lines of code. Each joint needs a time-series of positions and quaternions.

Position and orientation is part of the MotionVAE's output. The pose variable here is an array of size 267.

  • 0-3: Change in root position (x,y) and facing direction (yaw)
  • 3-69: Joint positions (x,y,z) for 22 joints
  • 69-135: Joint velocities (x,y,z) for 22 joints (not needed)
  • 135-267: Joint orientations in form of rotation matrices. Only first two columns of 3x3 matrix, so 6 numbers each joint.

Unfortunately I don't remember the order of the joints, so it'll take some time to plot the positions and find the corresponding bone in your rig. Hopefully this is helpful.

belinghy avatar Oct 22 '20 22:10 belinghy