Native H264 unable to decode last couple of frames of a video

[Wumpf]

The last 6 frames of this video never produce any images, getting stuck waiting for it indefinitely.

Is this the reordering logic in the ffmpeg listener gone wrong?

[emilk]

• Could be related to https://github.com/rerun-io/rerun/issues/8098

[emilk]

With https://github.com/rerun-io/rerun/pull/8104 we now get out all but the last two frames, which is an improvement, but not a fix 🤔

[emilk]

We've tried a lot of different tactics to get ffmpeg to flush:

• closing stdin (partially helps)
• ending the stream with NALU::EndSequence and EndStream (did not help)
• ending the stream with NalUnitType::AccessUnitDelimiter (did not help)
• https://stackoverflow.com/questions/50574086/ffmpeg-flushing-output-file-every-chunk/50576757#50576757 (did no help)

Interestingly enough, if I use ffmpeg to convert the file to a .h264 file and pipe that to ffmpeg, I do get all frames out:

cat file.h264 | ffmpeg -i - -probesize 32 -analyzeduration 0 -fps_mode passthrough -f image2 /tmp/frames/frame_%d.png

I used https://github.com/dholroyd/h264-reader/blob/master/examples/dump.rs to dump the contents, and there is no special NALUs at the end. The FFMPEG parameters aren't exactly the same as the one we set via ffmpeg-sidecar though, so worth digging more here.

We could try piping our nalu-stream to a file and compare it, for instance. Or read the .h264 file and piping it to ffmpeg-sidecar with our arguments.

---

This video also repros the problem, and has helpful timestamps

https://github.com/user-attachments/assets/7b2ead8a-d701-449b-90f1-aeb7b6cb9161