openscreenprotocol
openscreenprotocol copied to clipboard
w3c
Investigate message latency requirements for media sync
It would be helpful to have an upper bound for PresentationConnection message latency based on use cases like media synchronization or lip sync.
From https://www.w3.org/2017/11/07-webscreens-minutes.html#x08:
ACTION: @chrisn to propose latency ranges for media synchronization use cases (lip sync vs frame sync)
Also clock synchronization and clock skew correction were mentioned as potential uses for connection messages. The HbbTV media synchronization spec may provide some inspiration.
A rough model of different use cases vs accuracy requirements is Figure 1 in this paper, shown below:
the ITU performed strictly controlled tests with expert viewers and found that the threshold for [lip sync] detectability is -125ms to +45ms.
From: ITU-R BT.1359-1, Relative Timing of Sound and Vision for Broadcasting (PDF). ITU. 1998.
To close this out I think a note that implementations should strive to send/receive messages within the smallest latency bounds possible, and no greater than the lip sync threshold of 45ms.
I think that's fine. The original intent behind the suggestion was for OSP to potentially support tightly synchronized playback between agents (e.g., sub 10ms). That would require an additional timeline sync protocol to be added to OSP, which would need research and investigation, so I would leave as future work if there's interest, not a v1 spec feature.
ETSI TS 103 286-2 is the relevant spec for the HbbTV companion screen synchronzation capability, sections 4.3.4 and 4.3.5.
Thanks @chrisn. I will file a followup issue for the sub-10ms use case.
Filed https://github.com/w3c/openscreenprotocol/issues/325 for this.