Different results between demo_syncnet.py and run_syncnet.py

[leeeeeeo]

When using python demo_syncnet.py --videofile data/example.avi --tmp_dir /path/to/temp/directory, I get the same result as: AV offset: 4 Min dist: 6.742 Confidence: 10.447

But, when using run_syncnet.py, I get different result from above: AV offset: 2 Min dist: 7.093 Confidence: 9.238

Why is that? Thank you!

[leeeeeeo]

And I have another question: run_sync.py can output AV offset, Min dist and Confidence, but how to recover video without offsets?

offsets.txt are generated from run_sync.py, but it's never used after then.

[abhisheksgumadi]

Any update on this please?

[joonson]

run_pipeline.py runs the face tracking script and re-encodes the video using ffmpeg. I suspect that the re-encoding process is introducing an offset.

[abhisheksgumadi]

Hi @joonson , I also observed that the output of the Face Detector which produces the cropped video introduces a slight delay between the audio and the video. I know that the audio and the video do not have any delay in their original version where the face is not cropped.

Any idea which part of the code might be doing this as the results definitely get affected? With some guidance I am happy to debug it.

Thanks.

[Olivialovecode]

Hi, I also get results offset: 4, by running demo script. But I cannot tell what is the unit of the offset. Is it milliseconds?

[XinBow99]

Hi, I also get results offset: 4, by running demo script. But I cannot tell what is the unit of the offset. Is it milliseconds?

looks like frame index: offset < 0, the audio is faster than the video frame. offset > 0, the video is faster than the audio frame.