sitespeedio
Timestamp-overlay-added video misses some frames compare to the input video
One thing we noticed is that the timestamp-overlay-added video misses some frames compare to the input video.
I used this command ffprobe -select_streams v -show_packets <input_video> | grep pts_time to get the PTS of each frame from the input video (note that the input video isn't the original video, it's the orange-frame-removed video) , and I would expect that I can see each PTS in the output video, however this wasn't the case.
If you ran the output video frame by frame, you would notice that some PTS didn't exist. I don't have a great explanation for why this happens, however I think it's related to our videos are in VFR and the default muxer for mp4 is vsync 1 in ffmpeg, which means Frames will be duplicated and dropped to achieve exactly the requested constant frame rate.
I suggest we add a new arguments as -vsync 0 - Each frame is passed with its timestamp from the demuxer to the muxer. ) to the ffmpeg command, which in my testing fixes the missing frames issue. I'd also suggest that we change the output container from mp4 to mkv, because doing -vsync 0 to mp4 container will generate some warnings like
[mp4 @ 0x563627d34280] Non-monotonous DTS in output stream 0:0; previous: 6213566, current: 6213565; changing to 6213567. This may result in incorrect timestamps in the output file.
Note that, this isn't perfect, the generate video has some duplicate timestamps (I mean the real timestamp of the video itself, not the timestamp overlay we added). For example, frame 2, 3 and 4 maybe have the same timestamp. I don't see any side effects by this issue though, video players can still play the video correctly.
Hi @sefeng211 ah cool! Lets try doing that in a branch. For moving to mkv, I'm not up to date that works, do all browser support it?
oh, what I mean is the output video container for the timestamp-overlay-added video. We kept the rest as mp4.