webrtc-stats
webrtc-stats copied to clipboard
w3c
Add clarifications for when a frame is rendered
Several metrics (pause, freeze and inter-frame if #717 is merged) talk about incrementing or measuring "just after" a frame is rendered. We should expand this definition in a common place that all said metrics can point to.
For example is the before or after render buffering, is it affected by vsync etc. One might also consider what happens in the case that the same track is rendered in multiple places - not something you would do in practise, but it is possible to do, so the spec should clarify.
https://wicg.github.io/video-rvfc/#dom-videoframecallbackmetadata-expecteddisplaytime has a definition but it is rather vague (and I heard concerns about it not being accurate) but we should align these specs
Given the focus on webrtc, I think https://wicg.github.io/video-rvfc/#dom-videoframecallbackmetadata-presentationtime is probably the metric that makes most sense to align with - it doesn't have a "guess about the future" component.
Defined as "The time at which the user agent submitted the frame for composition".
So that would be after any potential buffering in the render pipeline? E.g. some feedback needed to tell webrtc that the frame that was previously output from webrtc is now finally being submitted for composition
@drkron Do you have opinions here?
Looks good to me. So this together with #717 would be a simpler way of determining various render frame rates. The alternative way that exists today is to use requestVideoFrameCallback. I wonder if a potential continuation is to also add various latency metrics, such as receive to render.
On that related note, if we want to go even further, we could consider a "media-playout" stats object for video. We did recently add "media-playout" stats object for audio: RTCAudioPlayoutStats. If it existed for video, it would make sense to put the playout related stuff there (separate issue if so and not entirely sure we want to, unlike audio, video is not the same path for multiple RTP streams)