VIBE icon indicating copy to clipboard operation
VIBE copied to clipboard

.fbx output is based on default smpl model not vibe output

Open shawkontzu opened this issue 4 years ago • 13 comments

It seems like the fbx_output.py only use the SMPL default male/female model as the input, and extract vibe pose output to create animations.

Is there anyway to create .fbx file based on VIBE mesh and 3D joints output? Any way to create a rigged model using VIBE output?

shawkontzu avatar Apr 05 '21 17:04 shawkontzu

Hello @shawkontzu If you like to test, I've made an addon last week that could help you to bring vibe data inside blender (if you use blender)

Its in beta stage, but maybe it can be usefull to you.

https://gumroad.com/l/mocap_import you can download it for free.

carlosedubarreto avatar Apr 05 '21 18:04 carlosedubarreto

Hi @carlosedubarreto thank you for your response. Does the add-on consider weight map? I see that a pure rig is running in your plug-in intro video, I am more looking for a full rigged model which includes the skin mesh + rig + weight map (how each vertex is impacted by hierarchical rig structure), similar to the default smpl fbx file.

shawkontzu avatar Apr 05 '21 18:04 shawkontzu

Oh. no... Run away from my addon... At the moment it is only to get the movement...only skeleton.

carlosedubarreto avatar Apr 05 '21 21:04 carlosedubarreto

@carlosedubarreto hi. i have a question:i can not get the result of demo.py,so i want to ask that the out put of demo.py is a file named *.pkl?I must get it to run fbx_output.py,because the *.pkl is the input of fbx_output.py python file??

liuhaorandezhanghao avatar Apr 22 '21 02:04 liuhaorandezhanghao

The question is i can not get the output of vibe,is it a file *.pkl?

liuhaorandezhanghao avatar Apr 22 '21 02:04 liuhaorandezhanghao

The question is i can not get the output of vibe,is it a file *.pkl?

Hello @liuhaorandezhanghao The output from vibe is a video with the pose estimation on top of the footage and the pkl file that you can use on fbx_output script to get an fbx with the mesh/animation, or use with the addon I've created (you will only get a skeleton and its animation)

carlosedubarreto avatar Apr 22 '21 09:04 carlosedubarreto

Thank you for your reply, can I enter a video by myself, the video contains human motion, get the output of vibe, generate fbx file, use this fbx file to drive other 3d models in unity, instead of SMPL, do you think this is OK? Did it happen?

On 04/22/2021 17:52,Carlos @.***> wrote:

The question is i can not get the output of vibe,is it a file *.pkl?

Hello @liuhaorandezhanghao The output from vibe is a video with the pose estimation on top of the footage and the pkl file that you can use on fbx_output script the get an fbx woth the mesh and animation, ou use with the addon I've created (you will only get a skeleton and its animation)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

liuhaorandezhanghao avatar Apr 22 '21 10:04 liuhaorandezhanghao

Anyway, the fbx model I got contains objects and motions, and I can use it to drive other 3d models, such as my own design

On 04/22/2021 17:52,Carlos @.***> wrote:

The question is i can not get the output of vibe,is it a file *.pkl?

Hello @liuhaorandezhanghao The output from vibe is a video with the pose estimation on top of the footage and the pkl file that you can use on fbx_output script the get an fbx woth the mesh and animation, ou use with the addon I've created (you will only get a skeleton and its animation)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

liuhaorandezhanghao avatar Apr 22 '21 10:04 liuhaorandezhanghao

I don't know how to do it in unity, but in blender you can retarget using autorig pro, Rokoko or other plugin like these

carlosedubarreto avatar Apr 22 '21 11:04 carlosedubarreto

@shawkontzu hi,Is there anyway to create .fbx file based on VIBE mesh and 3D joints output?

liuhaorandezhanghao avatar Apr 23 '21 01:04 liuhaorandezhanghao

AttributeError: 'NoneType' object has no attribute 'scenes' hi,Have you encountered this problem, thank you! When I run fix_output.py,meet this problem. Here is the code: def setup_scene(model_path, fps_target): scene = bpy.data.scenes['Scene']

########################### # Engine independent setup ###########################

scene.render.fps = fps_target

Remove default cube

#一般用bpy.data.objects表示所有物体的集合,Cube表示立方体

if 'Cube' in bpy.data.objects: bpy.data.objects['Cube'].select_set(True) bpy.ops.object.delete()

Import gender specific .fbx template file

bpy.ops.import_scene.fbx(filepath=model_path)

Process single pose into keyframed bone orientations

On 04/22/2021 19:24,Carlos @.***> wrote:

I don't know how to do it in unity, but in blender you can retarget using autorig pro, Rokoko or other plugin like these

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

liuhaorandezhanghao avatar Apr 26 '21 06:04 liuhaorandezhanghao

I dont remember having a problem with this code. You can watch this VIBE Fbx conversion tutorial and see if it helps you

https://www.youtube.com/watch?v=w1biKeiQThY

carlosedubarreto avatar Apr 26 '21 07:04 carlosedubarreto

Thanks !

On 04/26/2021 15:13,Carlos @.***> wrote:

I dont remember having a problem with this code. You can watch this VIBE Fbx conversion tutorial and see if it helps you

https://www.youtube.com/watch?v=w1biKeiQThY

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

liuhaorandezhanghao avatar Apr 26 '21 07:04 liuhaorandezhanghao