The audio-driven video image is flickering

[xiao-keeplearning]

Thank you for your excellent work! The flickering in the demo video driven by audio is quite severe. Is there any solution to optimize it?

[Zejun-Yang]

Thank you for your interest! The flickering issue may be attributed to the diffusion model, and we are actively working to address this in the future.

[Brawlence]

Is there any solution to optimize it?

Just pass it through FFMPEG again, using the time-domain median filter:

ffmpeg.exe -i .\input.mp4 -filter:v "tmedian=3" output.mp4

This will reduce flickering significantly

[oliverban]

Is there any solution to optimize it?

Just pass it through FFMPEG again, using the time-domain median filter:

ffmpeg.exe -i .\input.mp4 -filter:v "tmedian=3" output.mp4

This will reduce flickering significantly

Yes, but it will create glitches and artifacts like crazy! :P Cool trick for nearly static video though! 👍