Presentation time offset for MPEG DASH manifests not applied correctly for xHE-AAC and AC4 streams

[ronak2121]

Version

Media3 1.3.0

More version details

No response

Devices that reproduce the issue

Samsung S21+ with Android 14

Devices that do not reproduce the issue

None

Reproducible in the demo app?

Yes

Reproduction steps

Play attached media Wait for the first period to finish Watch the second period not honor the offset value (effectively restart playback from the beginning of the content)

Expected result

Media plays correctly - the first period and the second, honoring the offset correctly

Actual result

Playback starts, plays the first period, and then plays the second one, without honoring the presentationTimeOffset.

Media

We actively worked with the Exoplayer team here: https://github.com/google/ExoPlayer/issues/11000 to fix discontinuities with AC-4 streams. We just adopted the fix, but it looks to have broken applying the presentationTimeOffset correctly for both xHE-AAC and AC-4 audio streams. For some reason, it looks to be working with AAC-LC (which doesn't have I-Frames).

Based on our debugging, the commit the Exoplayer team made for our above issue is what caused this. Can you please look into what's going on, and please apply the correct fix?

We're not sure if you still have our AC-4 stream from the past, but you can still use that. We will also email you links to three clear streams - an AAC one (where presentationTimeOffset is being honored) and xHE-AAC and AC-4 where they are not anymore.

Bug Report

• [X] You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

[ronak2121]

FYI @rohitjoins and @tonihei

[ronak2121]

FYI we've emailed the adb bug report to the above email address. We will send you sample audio later today. Would appreciate a set of eyes from your team on this soon.

We still think our original proposed patch from: https://github.com/google/ExoPlayer/issues/11000 is the actual solution to the original problem rather than messing with timestamps....but open to hearing your guidance as well.

[ronak2121]

We've emailed an XHE-AAC stream as promised. An AC4 one will be forthcoming.

[ronak2121]

@rohitjoins, can you please advise on the next steps for this issue?

[ronak2121]

Hi @rohitjoins , is there any update?

[tonihei]

@ronak2121 The xHE-AAC link you shared via email results in HTTP 403 errors for me. Could you check that it's accessible or provide more details on how to access it? Not sure what the link is targeting, but the most important thing to have for testing is the actual DASH manifest referencing real media files that show the problem.

[ronak2121]

Just emailed you guys again with a new link.

[tonihei]

The link still gives 403 errors I'm afraid. Not sure if it expired or if there is another problem, but we can't really debug and look into the issue if it's not accessible to us.

[ronak2121]

Hi I emailed you a new link a few weeks ago. Please try it out.

[tonihei]

Thanks for the new stream, I was now able to access it. We don't monitor this email address directly, so please let us know here is you send new information to make us aware.

I was able to reproduce the issue and it is caused by the fact that the second period starts in the middle of a segment. This requires the player to preroll the data from the beginning of the segment. For formats without specific keyframes, there is an optimization that skips the samples early (for AAC-LC), so it seems to work correctly. For other formats like xHE-AAC and AC-4, we need to start decoding at earlier samples and can't skip them directly. Usually they would be skipped later, after decoding, but our DASH implementation didn't expect that a period starts in the middle of a segment and thus simply played out the entire segment again.

We can look into solving this, by note that even if we report it correctly in our pipelines, the player likely enters a very short buffering state at the transition where it needs to flush and restart the codec.

[ronak2121]

Hi Toni,

Thanks for getting back to me. What do you mean by the second period starts in the middle of a segment? Our segments are composed of multiple fragments, and we group those fragments together to form segments. From what I saw, you disregard whatever value we specify in the presentation time offset, to skip the segments before that timestamp. I double checked the math there several times, and the timestamp lines up to the correct sample/segment we should play from….and works fine with ShakaPlayer.

We have I-Frames on every ~1s fragment so I’m not following what you’re referring to there.

Ronak

On Jul 19, 2024, at 11:49 AM, tonihei @.***> wrote:

Thanks for the new stream, I was now able to access it. We don't monitor this email address directly, so please let us know here is you send new information to make us aware.

I was able to reproduce the issue and it is caused by the fact that the second period starts in the middle of a segment. This requires the player to preroll the data from the beginning of the segment. For formats without specific keyframes, there is an optimization that skips the samples early (for AAC-LC), so it seems to work correctly. For other formats like xHE-AAC and AC-4, we need to start decoding at earlier samples and can't skip them directly. Usually they would be skipped later, after decoding, but our DASH implementation didn't expect that a period starts in the middle of a segment and thus simply played out the entire segment again.

We can look into solving this, by note that even if we report it correctly in our pipelines, the player likely enters a very short buffering state at the transition where it needs to flush and restart the codec.

— Reply to this email directly, view it on GitHub https://github.com/androidx/media/issues/1440#issuecomment-2239495478, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANMUZ6UAJUVEDOVD6UDDQ3ZNEYQRAVCNFSM6AAAAABJCT6VCCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZZGQ4TKNBXHA. You are receiving this because you were mentioned.

[tonihei]

The first segment has a duration of about 20 seconds and is supposed to start at around 10 seconds. The presentation time offset is applied correctly in some parts of the pipeline, but we don't tell the decoder that it needs to discard some of the data again. This is because of the wrong assumption that every DASH period starts at the beginning of a segment.

We have I-Frames on every ~1s fragment so I’m not following what you’re referring to there.

Each fragment has its own sample index table, so the player still needs to parse through the entire file to find the right point.

[ronak2121]

Thanks Tony. I get it now. Please let us know when this is closer to being fixed so we can test it out with our streams.

[tonihei]

We submitted a fix for the main problem now (see commit above). But as highlighted earlier, there will still be a tiny buffering state at the transition point where ExoPlayer needs time to go through the decode-only samples it won't need for playback. Closing this issue as the remaining buffering is a wider performance improvement related to how we handle pre-roll samples at transition points.