Controlling how much the browser hits the server for data

[snblackout]

Description

I have rolled my own API to deliver the bytes for particular videos to the browser using videogular. There is quite high CPU usage on the server for each request from the browser. After enabling logging on the server, the browser is hitting the API A LOT for just small amounts of data.

Question

Is there any way to minimize the calls to the server? Have it buffer a bit more data so it hits it less?

FYI I'm using Videogular2 still, but since this is the now the maintained fork and read the code is identical for now, I'm posting here.

Attachments

Example of output of the logging on the server and how much the client is hitting it for small amounts of data.

2020-08-05 08:06:23.3364 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:23.5166 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:23.5941 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:23.6556 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:23.7211 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:23.7990 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:23.8911 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.0041 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.0783 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.2858 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.4032 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.4767 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.5101 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.5697 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.7397 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.8861 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:24.9731 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:25.1383 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:25.2612 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:25.3056 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:25.3878 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:25.4831 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:25.5650 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:25.9089 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:26.0172 Returning file stream for: /media/filename.mp4 2020-08-05 08:06:26.1075 Returning file stream for: /media/filename.mp4

[snblackout]

Just going to jump in before a reply comes in that I was doing some searching and came across a 4-part series that uses the Media Source Extensions (MSE) to help control buffering of the stream. I haven't dug into the videogular code here yet too much, but hoping someone that is familiar with the codebase can chime in to see what could be done to expose some buffering settings to allow some config.

I'm not using HLS or Dash of course here, it's a URL to a MP4 video, but the API is designed to allow "Range Processing" so the whole video is not returned, but only small portions of data that the client requests.

[shanehsu]

Could you clarify what the expected behavior is? Because it looks a lot like a streaming protocol and you want Videogular to download small parts of the same file using Range Processing. I don’t believe Videogular contains any code that fetches video asset. Either the browser would do that itself or it delegate to third-party libraries like hls.js or dashjs.

My guess is you wish for the every request to download a bigger range, is that correct? Did you roll your own API server that serves the file or are you using a standard HTTP server like NGINX?

[snblackout]

Yes I'm using .Net Core MVC API and below is the function wired to the endpoint to deliver the good stuff. Pretty simple stuff, not massive "stream protocol" in any regard. You could potentially spin this up rather quickly with your own code if you'd like. But the logging above was from this function that gets hit when the client reaches for more data.

public static Microsoft.AspNetCore.Mvc.FileStreamResult GetContentStream(System.IO.FileInfo filePath)
{
    try
    {
        System.IO.Stream stream = System.IO.File.OpenRead(filePath.FullName);

        var retValue = new Microsoft.AspNetCore.Mvc.FileStreamResult(stream, "application/octet-stream");
        retValue.EnableRangeProcessing = true;
        return retValue;
    }
    catch (Exception ex)
    {
        Logging.LogError(ex);
        return null;
    }
}

I'm not sure how Videogular determines to hand off things for other libraries like dashjs or hls.js to pick up and do what it needs depending on what is returned from the server for content type. I'm returning just a standard octet-stream content type, but like mentioned with Range Processing enabled so the entire stream isn't read into memory and returned.

I believe the octet-stream content type would be like any other URL pointing directly to an MP4 asset out there, but if the client is aware of the Range Processing, it will then kick in it's own partial streaming capabilities and request data as it plays.

Maybe it is dashjs doing the actual requests to the server, or maybe dashjs/hls.js gives up, then the URL is just then handled by standard HTML5 video tag? Unsure on this.

[snblackout]

Just FYI, I am using Videogular for Dash and HLS content in my project, just in this instance for this configuration I'm just pointing a URL to an MP4. I'd like to continue to use Videogular here as to not have to search for another player to achieve things. I'm willing to contribute for improvements where I can if I know enough about what is happening under the hood for Videogular.

[shanehsu]

I believe you're using something like,

<vg-player>
  <video [vgMedia]="media" #media src="/media/filename.mp4">
</vg-player>

in your component.

In this simplest form, the browser interacts with the HTTP server directly, Videogular simply provides a wrapper around the native HTMLVideoElement.

The browser, as you expected, would already use range processing. However according to this StackOverflow post, the browser does not specify the end of range which by default means the whole file.

You can either,

1. Roll your own buffer control scheme client-side. Something along the line using MediaSourceExtension, or
2. Make sure your server return bigger range by default. I believe this StackOverflow post offers a place to start. Though digging through the sources of ASP.NET MVC, those virtual functions offers plenty of opportunities to override just a few places to get the job done.

I am too lazy to run the examples, but providing the range header from the browser request/response would be helpful in determining which route is the easiest or worthwhile.

[snblackout]

Hello. Actually my tag looks like below. I'm doing some URL analysis to determine which platform the browser is running. Basically if it's not Apple, it uses Dash. But if the URL is just a .MP4 extension, I don't touch the URL, but the video still plays. I believe dashjs and hls.js just give up and the standard HTML5 video takes over.

<video #media1
	   [vgMedia]="media1"
	   [vgDash]="stream.getMediaStream().source"
	   [vgHls]="stream.getMediaStream().source"
	   [vgDRMToken]="stream.getMediaStream().token"
	   [vgDRMLicenseServer]="stream.getMediaStream().licenseServers"
	   (error)="onVideo1Error($event)"
	   id="singleVideo1"
	   preload="metadata"
	   crossorigin
	   playsinline>
</video>

But I did take the vgDash and the vgHls plus the DRM tags out and just used src with this testing. Same result.

I did some looking into your links. Yes, the browser requests which ranges it wants and nothing really the server can do I believe. I did change the way .Net opens and provided a much larger buffer size. #2 is older from previous .net core releases, the EnableRangeProcessing on latest .net core releases handles that now.

The problem still lay with the client requesting the correct ranges. When I increased the buffer size to 1MB on server, the browser initially did get a sizeable chunk first. But the client still hit the API once or twice every 3-5 seconds (much better). But it seems no matter what I've tried, once the playback time gets to about 30-40 seconds into the video, the browser immediately just starts hammers the API up-to 10 times per second. Inspecting the range values in the browser dev tools, it is getting at least 1MB each request (the video is only 100MB), so something within is freaking out and requesting data too often. But that doesn't make sense because I'm using Chromium and that shouldn't freak out like that.

I think my only choice is to use the Media Source Extensions, like I linked and you mentioned. I have no experience with that. Should that really be handled in my app or is that something easier implemented within videogular?

[snblackout]

Looking into the Wiki for MSE and it's worth noting quite a few players out there are to be said using Media Source Extensions: https://en.wikipedia.org/wiki/Media_Source_Extensions

dashjs and hls.js both are listed there as supporting Media Source Extensions. Again, just asking, but wouldn't Videogular be able to expose these MSE configurations to the user to set?

Found this for Dashjs, unsure if it applies to the library or their own player: http://cdn.dashjs.org/latest/jsdoc/module-Settings.html#~AbrSettings__anchor

[shanehsu]

In order to support streaming protocol not native to the browser/platform, e.g. HLS on Android, Dash on iOS (?). Libraries like HLS.js rely on MSE API to make HLS work on Android.

You will certainly be able to accomplish your goal with MSE. However, I don't see why Videogular would want to use MSE for static video file, much like in your use case. But this is really for the maintainer to decide.

I think you should mention the maintainer and talk about your use case and potential changes you want to add to Videogular in order to make this a part of Videogular.

Although you can always just make a custom directive like vgHls and handle your MSE logic within your custom directive.

I also like to add that, if the video file is served with NGINX or other HTTP server (and not with a C# Application), you might see a high reduce in processor utilization. But again, your use-case might require additional checks that is only doable in C#. This is just one man's opinion!

[IxquitilisSaid]

Hey, good afternoon! So just to be sure, for your use-case it would be enough to expose the config options (as the https://github.com/videogular/ngx-videogular/blob/master/libs/ngx-videogular/core/src/lib/interfaces/ihls-config.interface.ts)?

Honestly, I don't have that much experience with Dash or HLS but we can definitely make it work (plus it sounds pretty interesting). Do you have a repo I can use to test these things?

[IxquitilisSaid]

Stale?

[snblackout]

Hello. It would still be great to see these configuration options available. What would you need from me to continue forward? Right now my use-case is I have a content server that uses Wowza which supports HLS and Dash, depending on the platform (iOS, desktop, Android etc). So I would like to see the ability to set either framework configuration options.