Is this module the right one for me?

[venomone]

Hey folks,

I'm not sure if this module is the right one for me. At least I was not able to get it properly working. I have a Django app that runs celery (lambda like tasks execution) to build HLS streams. So basically a task takes a mp4 file from S3 processes this mp4 as HLS stream and uploads the HLS segments and *.m3u8 files back to S3. So far so good. I can also confirm that my conversion is done right, the stream plays wonderful under VideoJS (Multiple subtitles and Audio tracks are also working for me)

Now my HLS segments are placed on S3 and the master m3u8 playlist is referenced at my Database. Now I would like to start a streaming. This is working fine if I simple have the following location configured at my nginx.conf:

       location /streaming/ {
	       proxy_set_header Host $host;
	       proxy_set_header X-Real-IP $remote_addr;
           proxy_pass http://minio0:9000/Movies/;
       }

A view like this at my Django app to send the m3u8 to VideoJS:

def stream(request):
    playlist_url = str("http://localhost/streaming/teststream/master.m3u8") # normally this is a var.
    args = {
        'playlist_url': playlist_url,
            }
    return render(request, 'App/stream.html', args)

Then I simply do the following at my template:

<video-js id=player class="vjs-default-skin vjs-fluid" controls="true">
<source src="{{ playlist_url }}"  type="application/x-mpegURL">
  </video-js>
  <script src="{% static 'videojs/video.js' %}"></script>
  <script>
      var player = videojs('player',
          {fill: true},
          {preload: true});
      player.play();
  </script>

Nice, I have a working stream but how can I secure the segments I'm serving using the /streaming location? It would be nice to have all HLS segments served under a one-time link or some kind of cookie that says the user allowed the load the HLS segments.

Any hint on how to possibly resolve this would be awesome. Thanks in advance!

[erankor]

The purpose of this module is to perform the MP4->HLS segmentation on-the-fly, if you uploading to S3 pre-segmented HLS streams, you don't need this module. To put it differently - if you use this module, you don't have to do it in advance, you just store MP4s in S3, and the module takes care of transforming them to HLS/DASH/... upon request. This has the advantage of supporting multiple delivery formats without having to pay double for storage.

Regarding security, you can use modules like nginx's secure_link or our nginx-akamai-token-validate-module to verify that the requests to nginx have a valid token. But, you also need some way to propagate the token from the manifest requests to the segment requests. One way to do it, is to put the token on the path of the URL, e.g. http://domain/token/master.m3u8, if you do that and you use relative URLs in the manifest, the token will propagate. On the other hand, if you use a query param for the token, you can use our nginx-secure-token-module to embed the token in the URLs returned inside m3u8 manifests. If it helps you can look at this config for example - https://github.com/kaltura/nginx-vod-module/issues/923#issuecomment-456071406 Here, the manifest gets the token as query param, but it is then propagated on the path using vod_segments_base_url.

[venomone]

@erankor Thanks for your response, so I will remove the vod module again as I serve the segments already pre-encoded in HLS format. Okay! (this is because I don't want to waste CPU resources. I just want to encode stuff once.)

I already have nginx_secure_link compiled and nginx-secure-token-module also but I'm not sure how to glue this together ...

The config shown at Comment #923 you send shows that the user is serving is HLS segments locally (alias /home/filme/download/;) I don't do that. I just have the minio standing in some Datacenter and I have the web application hosted somewhere else...

To me it seems im simply not able to use secure link on a proxy_pass location

   location /streaming/ {
       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_pass http://minio0:9000/Movies/;
       secure_link $arg_st,$arg_e;
       secure_link_md5 "%LINK_SECRET%$uri$secure_link_expires";
       if ($secure_link = "") {
            return 401;
       }
       if ($secure_link = "0") {
            return 410;
       }
   }

this is how I create the link:

def cypher_link(data_attachment_secured):
    secret = settings.LINK_SECRET

    future = datetime.datetime.utcnow() + datetime.timedelta(seconds=120)
    expiry = calendar.timegm(future.timetuple())

    secure_link = f"{secret}{data_attachment_secured}{expiry}".encode('utf-8')

    hash = hashlib.md5(secure_link).digest()
    base64_hash = base64.urlsafe_b64encode(hash)
    str_hash = base64_hash.decode('utf-8').rstrip('=')
    cypher_link = (f"{data_attachment_secured}?st={str_hash}&e={expiry}")
    return cypher_link

Any suggestions?

[erankor]

I'm sorry, but this config has nothing to do with this module/other modules we developed. I don't provide support for nginx's secure_link module... there are probably enough guides out there that explain how to do it...

[venomone]

@erankor Sorry it was not my intention that you do my work. After some digging for a solution I will go with the following: I will manipulate the master.m3u8 playlist in that way that it only contains signed segments that videoJs can grab instantly from S3. by that we are eliminating the delivery vector via a proxy.

[dzungpv]

@venomone Can you share more info? I have the same purpose as yours. I guest that you use the module only for generate the .m3u8 file, and the player get the segment directly from the S3.

[venomone]

@dzungpv Well there are actually more problems that you are facing. For example, you cannot teach your player to generate signed S3 links except you want to share access credentials of your S3 with your Users (I don't think you want to do this or should do this at all!)

Basically the trick is the following. As you have no influence on what the player queries from S3 as this is getting references by a m3u8 playlist, you need to somehow make sure that each time the player requests a new segment from S3 that this request is authenticated. This is very, very important as otherwise your streams can easily get hot linked to other websites!

So how to solve this stupid problem? Basically there is only one real solution I was able to find.

1. Your Application needs a Backend API and a Frontend (e.g. Django for Backend and AngularJS for Frontend)
2. Your Backend is the only source that does user authentication using JWT (JSON Web Token)
3. Assuming you are using minio for S3 stuff, make sure that the bucket policy is set to "public". So that everyone can access your files (please read on)
4. As the bucket is public you need a 3rd party authentication module that works like a proxy in front of minio and does JWT validation of the tokens you have created at the backend. So yes you need some kind of shared secret or public key between your backend and the proxy that does the JWT validation (OpenResty can do this or NGINX Plus).
5. Now that all requests you send to the minio instance need a valid JWT token at the header, you can do something like that at your code:

<script>
  videojs.Hls.xhr.beforeRequest = function(options) {
    options.headers = options.headers || {};
    options.headers.Authorization = 'Bearer token';
    console.log('options', options)
    return options;
  };
  var player = videojs('my_video');
  player.ready(function() {
    this.src({
      ... bla bla some m3u8
</script>

Have a look at the following line: options.headers.Authorization This basically tells VideoJS to attach a Bearer token (The Users JWT token here) to every request referenced by the m3u8 playlist.... Problem solved with proper authentication of a user's segment request.

You can start here: https://github.com/TeslaGov/ngx-http-auth-jwt-module

Be aware that you have to build nginx/openresty yourself

Cheers