WebRTC to FFPlay Key Frame

[imtadashi]

Hi Sipsorcery Team. I was using WebRTC to FFPlay example and it works completely fine. But when I made two Peers using Sipsorcery and requested KeyFrame from one end, it doesn't work. Will forcing keyframe works only in Sipsorcery to Web App?

[ha-ves]

in that example, it sends the ask for keyframe (picture loss indication RTCP feedback) message to a web browser webrtc implementation which most likely handles it auto-magically.

You can manually force the keyframe on the source

// having a map is convenient and will cover for multiple stream(s) RTP session(s)
var VideoSourceTrack = new Dictionary<IVideoSource, MediaStreamTrack>();

Do add the tracks to the source RTCPeerConnection

var ThisPC = new RTCPeerConnection(...) { ... };
// ... foreach VideoSourceTrack, addtrack, onencodedframe -> track.sendvideo, etc.

Then configure the callback for key-frame request

ThisPC.OnReceiveReport += (ipEP, sdpMediaType, rtcpCompPkt) =>
{
    // Sanity check for feedback
    if (rtcpCompPkt.Feedback is var rtcpFB && rtcpFB != null
        && sdpMediaType == SDPMediaTypesEnum.video
        //Picture Loss Indication [RFC4585] a.k.a key frame requested
        && rtcpFB.Header.PayloadFeedbackMessageType == PSFBFeedbackTypesEnum.PLI)
    {
        VideoSourceTrack.First(vt =>
            vt.Value == ThisPC.VideoStreamList.First(v =>
                            v.RemoteTrack.IsSsrcMatch(rtcpFB.SenderSSRC)
                            && v.LocalTrack.IsSsrcMatch(rtcpFB.MediaSSRC)
                        ).LocalTrack
            ).Key.ForceKeyFrame();
    }
};

[imtadashi]

Thanks for the above code suggestion. I have a video source in this variable cameraSource. So I manually tried to call this cameraSource.ForceKeyFrame(), but it didn't work. The code you gave also do the same(cameraSource.ForceKeyFrame()) right?

[ha-ves]

Sorry, came back a bit too long.

It is up to the encoder to produce the keyframe.

It seems that the method for requesting the keyframe is not valid at the encoding control.

[sipsorcery]

The RTCP handling in this library is very immature. RTCP feedback reports are recognised but currently nothing is done with them. PR's welcome.