Local API: Implement SABR for VODs

[absidue]

Local API: Implement SABR for VODs

Pull Request Type

• [x] Feature Implementation

Description

Opening as a draft because when it works it works well but when it doesn't it currently errors with errors that I haven't been able to figure out how to resolve...

This pull request implements support for SABR playback with the local API for VODs, for live streams and post-live DVRs we use the manifests provided by YouTube. This also includes some code copied and adapted from shaka-player as they didn't export the MP4 and WebM segment index parsers outside of the package and I needed to make some changes to them.

Massive thanks to @LuanRT for being there for my silly questions about SABR and helping me debug and optimise this implemenation of it.

You will probably notice various /** @__NOINLINE__ */ comments dotted around the code in this pull request, I added those because I noticed that instead of properly inlining some of the functions or leaving them where they were, terser was inlining them as IIFEs, which means that instead of calling the function each time it was called, it would recreate the function first and then immediately call it, which is not ideal as this code is already adding more overhead to each segment fetch that shaka-player does. The /** @__NOINLINE__ */ comments tell terser to not even attempt to inline those function calls even if it thinks it is possible.

To give you an idea of what that dodgy function inlining looks like, here is a simplified example (you can see how recreating the function on each iteration of the for loop is not great):

let formats // = ...
const someArray = []

function someFunction(a, b) {
  for (const format of formats) {
    someArray.push((function (f) {
      return /** transformed format */
    })(format))
  }
}

Testing

Please try various videos and check that they play correctly in both DASH and audio modes.

Additionally to check that I didn't break things please also test some live streams and the legacy formats.

Desktop

• OS: Windows
• OS Version: 10
• FreeTube version: d875cc6642ac8277dad6cb42c7023ec6e7202c67

[github-actions[bot]]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions[bot]]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[PikachuEXE]

Got some different error handling loop with this branch I tried https://youtu.be/LoYf4522pzY (random video I want to watch probably nothing special) I saw some player icons appears/disappears repeatedly Here are the errors in console (partial, seems to repeat until I exit

[github-actions[bot]]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[PikachuEXE]

Got error in https://youtu.be/CVlDP0uNDiE (not sure if dev got the same

[absidue]

@PikachuEXE Did you get the invalid int32 error on the latest commit on this branch? If not it was likely already solved by https://github.com/FreeTubeApp/FreeTube/commit/89bd0c87dde1bd1a40a692c46038ebf6b2517213

[absidue]

As for the error in this comment (https://github.com/FreeTubeApp/FreeTube/pull/7145#issuecomment-2812076433), please expand the shaka-player data array and the FreeTube data objects before taking a screenshot. HTTP_ERROR is a wrapper, so without the details of what the actual error was, all I know is that there was an error.

[PikachuEXE]

invalid int32 error is from https://github.com/PikachuEXE/FreeTube/actions/runs/14505077935 which includes commit [Log recoverable networking errors as warnings](https://github.com/FreeTubeApp/FreeTube/pull/7145/commits/76cc43e7c7aa3713c1058d54a3e79122457b5828)

For the loop one I haven't reproduced yet, I will post the details if I encounter it again

[github-actions[bot]]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[absidue]

I can't reproduce the int32 error but I've now added logging for the data when the protobuf encoding fails, which should hopefully help us identify which field contains a decimal instead of integer number in your case.

[PikachuEXE]

A new one with sabr:// Video is https://youtu.be/-1HyNIV728A but I think it just happens randomly

[PikachuEXE]

OK error handling loop happens again

The URL in errors are (in order): sabr://audio?formatId=251-1736141248773268-ChEKBWFjb250EghvcmlnaW5hbAoNCgRsYW5nEgVlbi1VUw&init, twice sabr://video?formatId=397-1736162461584095-&resolution=480&init (until I open dev tool to screen cap error

[github-actions[bot]]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[Creature121]

Here are the errors I'm seeing in my console, in case it helps.:

[efb4f5ff-1298-471a-8973-3d47447115dc]

@Creature121 are you testing this PR?

[absidue]

Locking for the moment to limit the conversation to maintainers, to avoid off-topic comments like the one with screenshots that have nothing to do with this pull request.

[efb4f5ff-1298-471a-8973-3d47447115dc]

Duration of the video is 3:11 but it randomly stopped at 3:10 this is the second time i noticed that the video pauses 1 sec before ending on the most recent commit

https://github.com/user-attachments/assets/75d44795-1a58-4d33-ba52-6b03e3d1ecc4

Error:

https://github.com/user-attachments/assets/5c3d51b7-a0c4-445a-a3f4-11ae222964b7

[PikachuEXE]

Invalid int32 error after Invalid VideoPlaybackAbrRequest data No idea how to debug further

[absidue]

Looks like it's the same issue as @efb4f5ff-1298-471a-8973-3d47447115dc's second video, the bufferedRanges[1].endSegmentIndex is null. Not sure how we could solve it without coming up with a different approach for working out the buffered ranges.

[PikachuEXE]

I still got missing MEDIA_END https://youtu.be/5zO1RmxFR04 1440p pause near start and let it load (doubt it's video specific though

[PikachuEXE]

I found that with this video (but could be current time related, no VPN though) I can reproduce null endSegmentIndex quite easily (even I switch back to DASH it will still happen, especially near the end, even with 360p) https://youtu.be/9hDvWVJtljE

Let me see if I can reproduce this tomorrow and can add some debugging code

[PikachuEXE]

Added a bit of debug code

With https://youtu.be/9hDvWVJtljE I can reproduce the issue again (near the end of video It seems the segment index either empty itself at the end (but could also be due to we auto switch to legacy) Or find is bugged Will do more debugging

Before loading the end After error

Update 1: Researching https://github.com/shaka-project/shaka-player/blob/v4.14.10/lib/media/segment_index.js#L146 Update 2: It seems returning null for last end time is expected: https://github.com/shaka-project/shaka-player/blob/v4.14.10/test/media/segment_index_unit.js#L89-L95

[PikachuEXE]

I will apply the following patch and see how it works (Already tested in dev with https://youtu.be/9hDvWVJtljE

Update 1: Never got int32 again so far but still got empty response Update 2: I keep patching my own version but there is at least still 1 issue (randomly happening) that I have no idea how to workaround

In this issue parts are always

• SABR_CONTEXT_UPDATE
• SNACKBAR_MESSAGE
• NEXT_REQUEST_POLICY with empty playbackCookie (after encoded) & backoffTimeMs: 4000

[absidue]

Superseded by #8047