CVTHead
CVTHead copied to clipboard
HowieMa
edit headpose only and keep the expression
hi there, great project! in the inference.py --flame mode, in the demo that edits exp, it seems that the exp coef is randomly given rather than keep the source image expression for i in range(10): exp = torch.zeros(1, 100).to(src_img.device) # (1, 100) exp[0, 0] = 2 * i / 10 Is there any way to edit the head pose only and keep the input face expression the same?
You can keep the exp as a None value. i.e., if you don't pass the exp value, it will follow the original expression coefficient. The logic is here: https://github.com/HowieMa/CVTHead/blob/8f75b227fe35d3aec30920064ef7c9df5dcd8567/models/cvthead.py#L436
i see, then the drv_codedict['exp'] will keep the same as：
src_codedict = self.deca.encode(crop_src_img, use_detail=True)
however, when I input './examples/2.png' and print its src_codedict['exp'] the 50-dims FLAME exp coeff, it is very close to zeors like tensor([[ 0.6193, -0.2783, 0.2837, 0.3630, -0.0149, -0.3751, -0.2054, 0.2826,
-0.1413, 0.4739, 0.0610, 0.7318, 0.3283, -0.2038, 0.0750, -0.2088,
-0.2746, -0.0101, 0.1676, 0.3696, -0.0183, -0.1984, -0.0575, -0.1980,
-0.0062, -0.0128, 0.1302, 0.0862, -0.1784, 0.0785, -0.1404, 0.1140,
0.0795, -0.0530, 0.0306, -0.0548, 0.0012, -0.0391, 0.0938, -0.1831,
-0.4531, 0.1092, -0.0538, -0.0640, -0.1024, -0.0648, 0.0657, 0.1505,
0.1055, 0.0444]], device='cuda:0')
and the output pose.gif is close to neutral face and not consistent with the source image
is there any tricks in deca.encode and deca.decode?
Oh you also need to keep the original jaw pose. In the flame model, the model is always closed if you keep the default jaw pose.