Bento4
Bento4 copied to clipboard
axiomatic-systems
Frames shifted after mp4hls conversion
Win 10 mp4hls 1.1.0 r621
After running the cmd tool, mp4hls on a 10 second mp4, all of the frames are shifted.
frames gathered with ffprobe -show_frames
video: https://nofile.io/f/nmAU0z1gNOm/segment-0.mp4 input frames: https://pastebin.com/8Qu7fsw4 output frames: https://pastebin.com/FwYLcgQi
The first frame timestamp of the input is 0.0 The first frame timestamp of the output is 0.08
The last frame of the input is 9.96 The last frame of the output is 10.04
There should be a complete GOP at the end of the input file, as in the complete video the input was clipped from, it has a keyframe on the next frame (10)
Why would this happen? Can I avoid/negate this, or is this an issue with bento?
I'm trying to solve this problem: https://github.com/axiomatic-systems/Bento4/issues/173 by inserting key-frames at intended cue points, creating segments between cue points, and concatenating everything back together. It mostly works, but I'm seeing some hiccups with playback around the boundaries of the segments.
The 80ms shift in timestamp is what's specified in the input video you link to here. You can check the timestamp values that are signaled in the video stream, by using 'mp4info --show-samples --verbose ` With the file segment-0.mp4 you link to here, we can see that the first frame is specified to have a CTS value of 7200, which is 80 milliseconds (the media timescale is 90000). Now, the reason you're seeing different reported timestamps between the input and output is that the input also has a so-called 'edit list' ('elst' box in the content), which can provide a timestamp offset from the frame timestamps. In the input here, the 'elst' box has:
[elst] size=12+16
entry count = 1
entry/segment duration = 10000
entry/media time = 7200
entry/media rate = 1
So what's missing here is that the mp4hls tool should, but doesn't, use the 'elst' information to perform timestamp adjustments. In this case, it means that the video frames will have timestamps that are off by 80 milliseconds (which may or may not be visible in the end), but all frames are there in the output.
I'll try to fix that in the next release.
This might be something we may be able to help with. Can you point me toward the relevant code where this would be handled?
I ran into the same issue today with 23.976 framerate videos. The keyframes before:
0.000000
2.002000
4.004000
6.006000
8.008000
10.010000
12.012000
14.014000
16.016000
18.018000
and after mp4hls:
0.083422
2.085422
4.087422
6.089422
8.091422
10.093422
12.095422
14.097422
16.099422
18.101422
I assumed that ffmpeg somehow had changed the start time/offset but I noticed with ffprobe that mp4fragment made start: 0.000000 go to start: -0.023220 Could that have anything to do with it? Is there some kind of work around available until there is a fix available?
The 80ms shift in timestamp is what's specified in the input video you link to here. You can check the timestamp values that are signaled in the video stream, by using 'mp4info --show-samples --verbose ` With the file segment-0.mp4 you link to here, we can see that the first frame is specified to have a CTS value of 7200, which is 80 milliseconds (the media timescale is 90000). Now, the reason you're seeing different reported timestamps between the input and output is that the input also has a so-called 'edit list' ('elst' box in the content), which can provide a timestamp offset from the frame timestamps. In the input here, the 'elst' box has:
[elst] size=12+16 entry count = 1 entry/segment duration = 10000 entry/media time = 7200 entry/media rate = 1So what's missing here is that the mp4hls tool should, but doesn't, use the 'elst' information to perform timestamp adjustments. In this case, it means that the video frames will have timestamps that are off by 80 milliseconds (which may or may not be visible in the end), but all frames are there in the output.
I'll try to fix that in the next release.
Any updates on this issue?
