rome icon indicating copy to clipboard operation
rome copied to clipboard

Published 20 hours ago • verified publisher icon SamsungLabs

Jitter problem.

Open zhanglonghao1992 opened this issue 2 years ago • 5 comments

Hi～ When I tested it on videos, I found that there was obvious inter frame jitter in the result, as shown below.

https://user-images.githubusercontent.com/17874285/182549247-392a5e21-068a-4a59-8d24-13ebe19ccc50.mp4

Why does it cause such obvious jitter? Is there any way to optimize it.

zhanglonghao1992 avatar Aug 03 '22 07:08 zhanglonghao1992

Do you use DECA camera or a camera from the mobile-net?

khakhulin avatar Aug 03 '22 18:08 khakhulin

@khakhulin I use the default settings in infer.py.

zhanglonghao1992 avatar Aug 05 '22 06:08 zhanglonghao1992

So, it should be ok, with the natural jitter from the deca (see video on the site).

khakhulin avatar Aug 29 '22 13:08 khakhulin

@khakhulin Is there a way to reduce this jitter? What is the 'camera from the mobile-net' that you refer to?

nlml avatar Sep 14 '22 00:09 nlml

@nlml I just suggest using the original camera from the DECA, not the distiller version. I think if the jitter is a real problem. then you need to optimize the sequence of the keypoints itself. Since original DECA doesn't guarantee temporal consistency if you run it with per-frame expression transfer.

khakhulin avatar Sep 18 '22 22:09 khakhulin