Correlating RTCInboundRtpStreamStats with simulcast streams

[henbos]

On the sender side we have multiple "rids", but on the receiver side we only have a single stream. How to tell which stream it correlates to?

[henbos]

Can the CSRCs be used to figure this out, assuming the SFU sends CSRCs that hint the RIDs?

[fippo]

browsers don't receive simulcast and what they receive must look like a single RTP stream. See here for some of the guiding principles and/or have a :coffee: with that person

[murillo128]

@henbos are you talking about browser==simulcast=>SFU--stream->browser or browser==simulcast =>browser

[vr000m]

we are discussing the first scenario, not about receiving simulcast

[murillo128]

then incoming stream should be a normal RTP stream, either should have an ssrc or mid to correlate with as any other normal rtp stream, not sure if I understand the issue.

[fippo]

the sender might receive rtcp from the SFU which can show up as an inbound-rtp stream with remoteId set. This will have an SSRC but no rid iiuc

[vr000m]

The issue is about what the receiver knows about the source stream at the SFU.

There are SSRCs received by the SFU, which it may swap and send downstream to the receiving endpoint. i.e., SFU receives A_low and A_high and swaps them in and out when sending it downstream on SSRC B to the received.

The question is -- can the receiver know (via getStats) which stream is being downstreamed to it A_low or A_high?

visually, you could plot metrics (frameHeight, bitrates, etc) for SSRC A_low, A_high, and B and see which one is being forwarded.

[vr000m]

The question is -- can the receiver know (via getStats) which stream is being downstreamed to it A_low or A_high?

visually, you could plot metrics (frameHeight, bitrates, etc) for SSRC A_low, A_high, and B and see which one is being forwarded.

The answer to this may just be, this cannot be done because this could be signalled via several mechanisms and this is outside of scope of webrtc-pc to mandate.

[vr000m]

One proposal:

I have not thought of this enough, and my first thought was why not use CSRC to indicate which source stream is being carried, at any given time there would be just one, but I dont know if the SFUs are presently using CSRCs to indicate something else.

(in the case of MCUs, they MAY use a set of CSRCs to represent which streams are mixed but here we are not mixing but switching so ...)

[murillo128]

I thought it was about the value remoteId of the stats from the incoming stream on the same transceiver in which it was sending simulcast.

Regarding correlating the stream received by the "browser 2" from the SFU with the simulcast stream sent by "browser 1" to the SFU, I don't think it will be possible. The SFU will not negotiate the rid values with the "browser 2" so even if they are not removed, the browser should ignore it. The SFU could keep the SSRC of the original sender (but that will be prone to errors due to potential SSRC collisions between peers).

Note also, that the SFU may reuse same track to send different participants and not have to renegotiate each time a new participant speaks.

[murillo128]

@vr000m I don't think any SFU is actually using the CSRCs at all

[vr000m]

@murillo128 Since for all practical purposes the RTP Session terminates at the SFU -- we cannot use SSRCs across RTP Sessions (I should check the taxonomy draft for the proper terminology) and any idea to reuse it across sessions is brittle when there are a large number of participants because there is high probability that the SSRCs would collide and doing conflict resolution for SSRCs outside of the session will eventually fail.

[vr000m]

@vr000m I don't think any SFU is actually using the CSRCs at all

:) then technically -- it could be used 🕺

[murillo128]

@vr000m even if they could be used, how would you correlate the information with the appropiate participant and the with it's ssrcs? It would require external coordination.

Anyway, if it is just exposing the CSRCs values in the stats and not adding any other logic, why not. (is it not already done for MCUs?)

[vr000m]

@vr000m even if they could be used, how would you correlate the information with the appropiate participant and the with it's ssrcs? It would require external coordination.

Yeah, it would require external co-ordination, but that is not heavy lifting IMO. because you could do that easily in post analysis.....

1. I dont think it should be an issue if an upstream SSRC is reused on the downstream, because it easy to exclude from the logic. SSRC collision would not allow upstream and downstream reuse, so the CSRC SSRC matching a down or upstream SSRC would mean it is external.

2. Two independent endpoints could use the same SSRC, then it would be hard to disambiguate which endpoint it it. I think this may be the issue/challenge of using CSRCs

[jan-ivar]

(I deleted a comment as I got confused about remote stats in the SFU case).

the sender might receive rtcp from the SFU which can show up as an inbound-rtp stream with remoteId set. This will have an SSRC but no rid iiuc

Thanks, this helps clarify. So we get rtcp data for each layer?

So "remote-inbound-rtp" on the sender will come from the SFU—rather than the ultimate recipient—one for each layer? Then once we fix https://github.com/w3c/webrtc-stats/issues/395 we can do stats.get(stat.localId).rid.

And on the (ultimate) receiver, "remote-outbound-rtp" will come from the SFU—not the original sender—and there's only one layer ever. No rid here. Correct?

[vr000m]

And on the (ultimate) receiver, "remote-outbound-rtp" will come from the SFU—not the original sender—and there's only one layer ever. No rid here. Correct?

Yes, the ultimate receiver is not aware of simulcast at all. All the switching between tracks at the SFU are hidden from the downstream receiver. To the receiver it is just one stream.

[henbos]

@vr000m Is there anything to be done here for TPAC?