webrtc
webrtc copied to clipboard
webrtc-rs
Add AV1 Support
Pion added av1 support:
https://github.com/pion/webrtc/pull/2071 https://github.com/pion/webrtc/pull/2176
This would be nice to have, but I don't expect it to be priority for those working on the project at the moment.
If it's something you are interested, it'd be great if you could contribute it
@k0nserv Absolutely! I did some experimenting with just supporting the mimetype / codec to get AV1 flowing and it was pretty simple. Going to post a PR for that shortly.
Looking to eventually add support for the dependency descriptor rtp header extension (AV1 SVC) but don't have the cycles for it at the moment =]
Reference: https://aomediacodec.github.io/av1-rtp-spec/
Also: https://github.com/pion/rtp/pull/172
I wonder what the current state of AV1 support is. I added the codec but apparently I also need a payloader for it?
It depends. If you're only forwarding RTP packets there should be no need for a payloader, but if you're using TrackLocalStaticSample, then yes a payloader is needed.
If you want to contribute it that would be very welcome, the details on how to are available in RTP Payload Format For AV1 (v1.0)
I'm taking a stab at implementing the AV1 payloader today. My plan is to base it on pion's payloader. Any help or guidelines would be greatly appreciated.
I'm trying to understand the scope of this work. Do you think that implementing the AV1 payloader and returning it from payloader_for_codec would be enough to stream AV1 using TrackLocalStaticSample? Or, is there anything else that I would need to implement?
An update on this: I ported pion's payloader but Chrome is still not happy with it, it keeps sending PILs and not displaying anything. I have proof that the raw encoded frames are correct (I dumped it into a file and I can ffplay it). Still trying to figure out why Chrome is not able to decode the stream though.
I'm going to step debug a local Chromium build to investigate but any tips on how to debug the decoder side would be helpful.
Okay so Chrome is printing these warnings:
video_rtp_depacketizer_av1.cc(240)] Malformed fragment size 242278315 is larger than remaining size 1173 while reading obu #2/0
@k0nserv do you know if I have to implement something for the "packetization" part too? or does this mean that my payloader port is incorrect?
The pion impl has an Unmarshal function that I didn't port assuming it's only used for receiving AV1 (as opposed to sending) but I'm not sure that assumption is correct.
I'm not really active in this project anymore, that said you should only need packetization. It looks like you've made some mistake with a length field, e.g. a an overflow/underflow, wrong endianess, or something with the offsets in the RTP payload being wrong
Hi all, I've fleshed out an implementation of the AV1 payloader by porting over Chromium's C++ implementation in this PR: https://github.com/webrtc-rs/webrtc/pull/471.
I believe the problem with @juancampa 's approach is that pion's payloader assumes the payload always contains exactly one OBU, which is apparently not true. Chromium's implementation handles parsing multiple OBUs from the payload and then packetizing accordingly.
The current implementation does have some caveats as noted in the PR description, such as potentially sending (marking) redundant key frames. However, I have tested (as being used in my own project) that the packets can be successfully decoded by Chrome with no apparent issues. Therefore I think this is a good start.
@k0nserv @juancampa @billylindeman Feel free to give it a try, and any feedback is welcomed :)
