rerun-io
Native Video support
We'll soon have experimental video support in the web viewer (https://github.com/rerun-io/rerun/issues/7271).
The next step is to get that working on native too.
Progress
Experimental support using rav1d can be found in:
- https://github.com/rerun-io/rerun/pull/7557
TODO
- [x] https://github.com/rerun-io/rerun/pull/7563 (required by rav1d)
- https://github.com/rerun-io/rerun/issues/7575
- [ ] Error handling and cleanup of https://github.com/rerun-io/rerun/pull/7557
- [ ] Check memory use
- [ ] Move chroma-upsampling on GPU
- [ ] Update the video support matrix docs
Current plan is to use https://github.com/rerun-io/rav1d-rs (a fork of dav1d-rs which replaces the underlying dav1d library with rav1d) to decode AV1 video.
We're uncertain about how to support other codecs, at least VP8/9 should be easily doable in a similar way as AV1, but H.264/265 is a bit of a special case.
(Note that I made the same mistake as in re_mp4 and uploaded a bunch of binary files to rav1d-rs, which need cleaning up via history rewriting and either be turned into LFS files or removed)
@jprochazk I summarized my understanding of rav1d-rs in this issue - feel free to edit/comment:
- https://github.com/rerun-io/rerun/issues/7575
There's always more codecs we'd like to support, but the ones listed here are all in!
For a full overview what we support check https://rerun.io/docs/reference/video#codec-support
