SMPL translation parameters ?

[anilkunchalaece]

Hi,

How can we calculate translation parameters ? The output file consist of pose , shape but I don't find any translation parameters.

[carlosedubarreto]

Hello @anilkunchalaece I've been analysing vibes pkl result file and for me, it seems that probably to add the translation, it would be nedded to join the camera movement with the hip bone pose.

You can see more about these parameters that are in the PKL file in this link

https://github.com/mkocabas/VIBE/blob/master/doc/demo.md#output-format

[anilkunchalaece]

Hi @carlosedubarreto Thanks for response. could you please explain bit more about it ? and provide references to it if any ? I guess I need to join orig_cam with hip bone pose joints2d / joints3d right ?

[carlosedubarreto]

Unfortunetely I can't explain more, because I wasnt able to test it. I just thought about it. I was also trying to make the body move, instead of being still, and the only way that came to mind was using both root bone (hip in this case, i guess) and the camera position. But I dont know how to use them together to make the final result of the body moving..

"I guess I need to join orig_cam with hip bone pose joints2d / joints3d right ?" Yeah, I think that is the way.

[anilkunchalaece]

From hmr - 60

    cam_s = cam[0]
    cam_pos = cam[1:]
    flength = 500.
    tz = flength / (0.5 * img_size * cam_s)
    trans = np.hstack([cam_pos, tz])

Are these what we are looking for ? Same also referenced here

[nosy-b]

@carlosedubarreto I managed to get the body movin (great song), using orig_cam = output[1]['orig_cam'] from the pkl and computing perspective coordinates using the function you mentioned. (camera parameters in original image space (sx,sy,tx,ty) not the cropped one). My function:

img_size = 1280
flength = 500
outPutCoords = []
orig_cam = output[1]['orig_cam']

for cam in orig_cam:
    cam_s = cam[0:1]
    cam_pos = cam[2:]
    tz = flength / (0.5 * img_size * cam_s)
    trans = np.hstack([cam_pos, tz])
    outPutCoords.append(trans)

Then I rearranged to my coord system in WebGL. See here, with a special video I made to understand easily axis.

You can't see a dirty draft here with bad synchro... http://whenistheweekend.com/webgl/vibeViz/gltf.html

[carlosedubarreto]

Really cool @nosy-b !!!!! I'll bookmark it for a test.

thanks a lot for sharing your discovery! 😍

[tegusi]

For classical intrinsic and extrinsic matrix, the following solution works well.

cam = dicts['orig_cam']
cam_s = cam[0:1]
cam_pos = cam[2:]
flength = w / 2.
tz = flength / (0.5 * w * cam_s)
trans = -np.hstack([cam_pos, tz])
camera_data['color_focal_length'].append(np.array([w / 2, w / 2]))
camera_data['color_center'].append(np.array([[w / 2, h / 2]]))
camera_data['c2w'].append(np.eye(4))
camera_data['c2w'][:3,3] = trans

[surheaven]

@carlosedubarreto I managed to get the body movin (great song), using orig_cam = output[1]['orig_cam'] from the pkl and computing perspective coordinates using the function you mentioned. (camera parameters in original image space (sx,sy,tx,ty) not the cropped one). My function:

img_size = 1280
flength = 500
outPutCoords = []
orig_cam = output[1]['orig_cam']

for cam in orig_cam:
    cam_s = cam[0:1]
    cam_pos = cam[2:]
    tz = flength / (0.5 * img_size * cam_s)
    trans = np.hstack([cam_pos, tz])
    outPutCoords.append(trans)

Then I rearranged to my coord system in WebGL. See here, with a special video I made to understand easily axis.

You can't see a dirty draft here with bad synchro... http://whenistheweekend.com/webgl/vibeViz/gltf.html

Hello. What do img_size and flength stand for? My image size is 1920*1080 and the focal length is 29mm, so what should be the img_size and flength?

[mayank64ce]

@anilkunchalaece did find a way to get 'trans' parametres for vibe_output.pkl ?

[anilkunchalaece]

@mayank64ce check this https://github.com/mkocabas/VIBE/issues/172#issuecomment-759501825