Zhenhui Ye comments

Results 41 comments of


                                            Zhenhui Ye

请教如何微调头部姿势

Hi, can you describe what is the `lambda_dynamic`? I wonder is it `lambda_ambient` since I cannot find a variable named `lambda_dynamic`

Error in running custom target video

Hi, it seems that in the 430th frame of the training video, the landmark detector has not find a face. Please check the 430th frame of the video, and consider...

When will the Instant Motion2Video Model be released?

Hi, in GeneFace++, we have updated many fundamental improvements in the codebase (such as 3DMM extraction). Ideally, we can upload it in a new repo by the end of Feb,...

Errors in processing May.mp4 (Step 3): The code extract_3dmm.py under process_data.sh fails to calculate 3DMM throwing serveral errors.

Hi, can you provide more details about the error so that I can help you out?

Errors in processing May.mp4 (Step 3): The code extract_3dmm.py under process_data.sh fails to calculate 3DMM throwing serveral errors.

hi, I have not confronted with this error, could you please print the w0, h0, s, t[0], t[1] as raised by: File "/root/GeneFace-1.1.0/deep_3drecon/util/preprocess.py", line 205, in align_img trans_params = np.array([w0,...

Errors in processing May.mp4 (Step 3): The code extract_3dmm.py under process_data.sh fails to calculate 3DMM throwing serveral errors.

Hi, it seems to be caused by the downloading of DeepSpeech ckpt was failed, so the process.sh just skip extracting the deepspeech. Maybe you could run this commandline to rerun...

Errors in processing May.mp4 (Step 3): The code extract_3dmm.py under process_data.sh fails to calculate 3DMM throwing serveral errors.

can you also post the error log here?

头部忽大忽小

你好，感谢你的指出！这个问题我们最近也发现了，也是定位在动作表征上面。目前猜测是因为3DMM的预测是由神经网络得出的，所以会有抖动。而我们的audio2motion是以这个预测的3DMM作为训练样本的，因此训练出来的模型预测的3D landmark也会有这种抖动感。目前这个问题难以很快解决，等我们找到合适的根治方法，会第一时间在这里更新代码。权宜之计是你可以把嘴唇以外的部位做smooth，应该可以部分缓解这个问题。

头部忽大忽小

理论上是可以的，3dmm和这个faceverse本质上讲都是基于统计模型的动作表征。不过你可能需要自己实现一下提取这个特征的代码。