What is the timestamp value of the VideoFrame/AudioData from a remote track?

[murillo128]

I have not been able to find if there is any specification about what is the value of the timestamp attribute of an VideoFrame or AudioFrame from a remote MediaStreamTrack recevied by a webrtc peerconnection.

Is is specified anywhere?

[alvestrand]

Interestingly, this has come up in other contexts today. I think we need to find an answer (preferably a consistent one). Have you observed this value in the current state of implementation?

[murillo128]

It seems that it is using the elapsed time since the first received frame (i.e. relative timestamp since the beginning of the call, starting at 0)

https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/webrtc/convert_to_webrtc_video_frame_buffer.cc;drc=047c7dc4ee1ce908d7fea38ca063fa2f80f92c77;l=347

https://jsfiddle.net/ytqn98eb/

[youennf]

This is being discussed in https://github.com/w3c/mediacapture-transform/issues/96. It seems good to me for each spec defining a track source to define how to extract a VideoFrame from it.

As of webrtc video frames, the closest I could get is the definition of captureTime: For video frames coming from remote source, the capture time is based on the RTP timestamp of the frame and estimated using clock synchronization. This is best effort and can use methods like using RTCP SR as specified in RFC 3550 Section 6.4.1, or by other alternative means if use by RTCP SR isn’t feasible.

[dontcallmedom-bot]

This issue was discussed in Joint Media/WebRTC WG meeting at TPAC – 26 September 2024 (What is the timestamp value of the VideoFrame/AudioData from a remote track?)