Finding associated audio and video tracks?

[WeichengSu]

According to the definition of estimatedPlayoutTimestamp, we can use audioTrackStats.estimatedPlayoutTimestamp - videoTrackStats.estimatedPlayoutTimestamp to get the AV sync. https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-estimatedplayouttimestamp

I have been using trackIds in RTCMediaStreamStats to find the associated audio and video tracks. https://w3c.github.io/webrtc-stats/#idl-def-rtcmediastreamstats However, it's made obsolete. What's the recommended way to find the associated audio and video tracks without RTCMediaStreamStats?

[alvestrand]

When you say "associated", do you mean the tracks that the platform will try to keep in sync (by including them in a MediaStreamTrack)? Applications are trying to keep track of A/V differences between tracks that aren't in a MediaStreamTrack - in that case, the stream stats wouldn't help you.

Can you explain more about what the use case is that you need this association for?

[WeichengSu]

Yes. I meant the tracks that the platform should try to keep in sync. We want to monitor the av sync of the two tracks (using estimatedPlayoutTimestamp https://w3c.github.io/webrtc-stats/#dom-rtcinboundrtpstreamstats-estimatedplayouttimestamp)

[vr000m]

are these tracks using the same CNAME? otherwise clocks might be different and hinder any A/V sync calculations.

[alvestrand]

According to spec, all tracks sourced from the same WebRTC implementation use the same CNAME.

[alvestrand]

Two possible changes:

• Add a MediaStreamIdentifier array to "sender" and "receiver" object
• Un-obsolete the MediaStreamStats object.

The latter is unpalatable because it is defined as pointing to (obsolete) Track stats objects.

[henbos]

The way to do this now is to listen to "RTCPeerConnection.ontrack". But if it is useful to tell this from a getStats call, we could add them like @alvestrand said above.

[henbos]

Proposal: add streamIdentifiers (DOMString array) to "sender" and "receiver"