androidx
period@start time is missing in presentationTimesUs calculation of EventStream
Version
Media3 main branch
More version details
We have encountered with an issue with the Event@presentationTime calculation in Exo Player. Based on the Exo Player logic, the event’s presentation time is calculated as:
Event@presentationTime - EventStream@presentationTimeOffset
Here is the manifest for reference:
<Period id="173" start="PT14H17M58S">
<EventStream schemeIdUri="metadata-update" timescale="1000" presentationTimeOffset="51478000">
<Event presentationTime="11211" id="2054">...</Event>
...
This results in a presentation time of -51466789, which is way before the Period's start time and this appears incorrect.
Referencing the MPD events timing model in the DASH spec, The formula for the event start:
The ST of an MPD event, relative to period start time of Period containing the Event, can be calculated using values in its EventStream and Event elements:
ST = PST + (PT - PTO)/TS
where:
— PST is the Period start time of the Period containing the event stream.
— PTO is the presentation time offset of the Event Stream provided by EventStream@presentationTimeoffset.
— TS is the value of EventStream@timescale.
— PT is the value of EventStream@presentationTime.
Which is: PST = PT14H17M58S = 51478S PT = 11211 PTO = 51478000 TS = 1000 Based on the given formula: ST = PST + (PT - PTO) / TS The calculated Event Start Time is 11.211 seconds.
However, ExoPlayer provides a presentation time of -51466789, which indicates that ExoPlayer does not appear to follow the latest DASH specification (Edition 5).
Dash Spec: https://standards.iso.org/ittf/PubliclyAvailableStandards/c083314_ISO_IEC%2023009-1_2022(en).zip
I checked the Exo Player code and found that parseEvent() API of DashManifestParser, It calculates the presentationTime time as below:
long presentationTimesUs =
Util.scaleLargeTimestamp(
presentationTime - presentationTimeOffset, C.MICROS_PER_SECOND, timescale);
Which does not follows the latest Dash Spec(Edition 5).
Reference:
Devices that reproduce the issue
Any Android Device
Devices that do not reproduce the issue
No response
Reproducible in the demo app?
Yes
Reproduction steps
Play a live event which has the EventStream with EventStream@presentationTimeOffset and EventStream@presentationTime. When we get the EventStream.presentationTimesUs it returns the incorrect PT time.
Expected result
While creating the EventStream instance, We should also add the period@start time in presentationTimesUs to follow the DASH specification.
Actual result
While creating the EventStream instance, We are not adding the period@start time in presentationTimesUs.
Media
NA
Bug Report
- [x] You will email the zip file produced by
adb bugreportto [email protected] after filing this issue.
Thanks for your report and the detailed explanation!
Your reasoning assumes that the value calculated in parseEvent is the ST from above, which I'm not convinced this is true. Instead this simply calculates the presentation time of the event within the period which is the right part of the equation above:
PT - PTO / TS
This is parsed and then set into the Pair<Long, Event> that is returned by parseEvent. My interpretation is that the calculated value presentationTimeUs is the start time of the event within the period (relative to the period start) and a period in ExoPlayer always starts at position 0.
I think that the terms period start time in Figure A3 from above, refers to the start time of the period within the stream. Whereas the start of the period in ExoPlayer is always 0.
This kind of simplifies the equation from above because in the sense of PST=0.
When looking into MetadatRenderer I mean that the code seconds my reasoning from above:
In MetadataRenderer.getPresentationTimeUs, the actual presentation time relative to the period start is calculated by subtracting outputStreamOffsetUs from positionUs. positionUs is the current render position.
positionUs - outputStreamOffsetUs
The render position of the period start is set when MetadataRenderer.onStreamChanged is called and offsetUs is passed in that then is assigned to outputStreamOffsetUs used by getPresentationTimeUs(positionUs).
Further, from the equation above and the manifest values presentationTimeOffset="51478000" and presentationTime="11211" (as in 'offset larger than pt') the start of the event always lands at a time that is before the start of the period which sounds a bit odd.
From this theoretical reasoning I think the calculation in DashManfiestParser is correct within the ExoPlayer logic.
However, besides theory: Do you see any problems with such a Dash live stream and events emitted by ExoPlayer that differs to the behavior of another dash player like [1]?
[1] https://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html
Further, from the equation above and the manifest values
presentationTimeOffset="51478000"andpresentationTime="11211"(as in 'offset larger than pt') the start of the event always lands at a time that is before the start of the period which sounds a bit odd.From this theoretical reasoning I think the calculation in
DashManfiestParseris correct within the ExoPlayer logic.
@marcbaechinger while I cannot speak to the reference anchor points that Exo uses to perform calculations I wanted to add some additional information regarding the DASH spec itself and why this sort of data may occur.
7.2.4 Repeating events
7.2.4.1 Repeating event instances in one Period
Events in the same Period should not be repeated. Since the player sees all events of a Period, there is no benefit for
repeating events in that Period.
7.2.4.2 Repeating the event cross Period
Events of the previous Period that continue into the next Period (i.e. are still active) should be repeated at the next
Period. The next Period should include a corresponding EventStream and Event elements. The EventStream element of
an event continuation should include an @presentationTimeOffset attribute.
7.2.5 Event timing considerations
The considerations are as follows:
1. An EventStream element may have an @presentationTimeOffset, an Event with a presentation time
matching the value of the @presentationTimeOffset will align to the Period@start. By setting an
event’s presentation time to a value smaller than the @presentationTimeOffset, an event that starts
before Period start time can be signalled.
2. In some common cases, the @presentationTimeOffset of an EventStream Element matches the
@presentationTimeOffset of the Adaptation Sets the events apply to (after timescale conversion). It is
recommended to use such common timeline for events
Notably: "By setting an event’s presentation time to a value smaller than the @presentationTimeOffset, an event that starts before Period start time can be signalled."
A specific application of this occurs in the SCTE-214 specification:
3. EventStream@presentationTimeOffset should be present. When EventStream@presentationTimeOffset
is present and its value should be set such thatEvent@presentationTime of an event starting before the
start of the current period would still be a non-negative number.
Note 1: the above allows expressing events happening in the past without keeping periods for this
purpose. As a result, the receiver may receive the same SCTE DASH Event element twice, in different
periods, and is expected to handle this.
Note 2: It is possible to do “epoch locking” and start the event timing from the POSIX Epoch time,
January 1, 1970 00:00.00Z.
This implies that EventStream@presentationTimeOffset set to the number of ticks since the Epoch
till the start of the period, and EventStream@presentationTime is set to the number of ticks from the
Epoch till the event start as calculated from the SCTE 35 cue message. Note that the condition
EventStream@presentationTimeOffset > Event@presentationTime
is legal and indicates an event starting prior to the start of the current period.
The key quote being: "Note that the condition EventStream@presentationTimeOffset > Event@presentationTime is legal and indicates an event starting prior to the start of the current period."
However, besides theory: Do you see any problems with such a Dash live stream and events emitted by ExoPlayer that differs to the behavior of another dash player like [1]?
[1] https://reference.dashif.org/dash.js/nightly/samples/dash-if-reference-player/index.html
Speaking of dash.js, this is the logic they implement for handling Event/EventStream attributes to calculate the event time: https://github.com/Dash-Industry-Forum/dash.js/blob/9cddbe588f982b905a4caede615a7e2cac7fec85/src/dash/models/DashManifestModel.js#L1110-L1111, which conforms to the formula mentioned in the issue description.
Adding a reference to dash.js: https://github.com/Dash-Industry-Forum/dash.js/blob/9cddbe588f982b905a4caede615a7e2cac7fec85/src/dash/DashAdapter.js#L493
At line no. 493: It adds the periodStart time to calculate the final PT.
// The PTO in voRepresentation is already specified in seconds
const presentationTimeOffset = !isNaN(voRepresentation.presentationTimeOffset) ? voRepresentation.presentationTimeOffset : !isNaN(eventStream.presentationTimeOffset) ? eventStream.presentationTimeOffset : 0;
// In case of version 1 events the presentation_time is parsed as presentation_time_delta
let presentationTimeDelta = eventBox.presentation_time_delta / timescale;
let calculatedPresentationTime;
if (eventBox.version === 0) {
calculatedPresentationTime = periodStart + mediaStartTime - presentationTimeOffset + presentationTimeDelta;
} else {
calculatedPresentationTime = periodStart + presentationTimeDelta - presentationTimeOffset;
}
...
...
event.calculatedPresentationTime = calculatedPresentationTime;
Given PTO = 51478000 and PT = 11211, when we add PST = PT14H17M58S = 51478 seconds to it, the calculated presentation time (PS) would be 11.211 seconds, as expected.
Thanks everyone.
Can you please provide me with a test fixture that exhibits a problem with events being emitted wrongly or not emitted and describe the malfunctioning so we can look into this.