Cannot play SAMPLE-AES nor Segment AES fMP4 using VHS 2.2 non-native

[RafalLukawiecki]

Description

When playing back fMP4 HLS H264 stream that has both audio and video AES-Sample or AES-Segment encrypted using Apple mediafilesegmenter an uncaught exception is thrown in non-native playback using Chrome and Firefox. Safari plays back without any problems.

Steps to reproduce

1. Using hosted VHS 2.2.0 player at https://videojs-http-streaming.netlify.app/ try playing Sample encrypted stream at https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-sample-aes/m.m3u8
2. Try playing Segment encrypted https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8

Results

Expected

Normal playback.

Actual

Uncaught exception, and player hangs indefinitely.

Error output

Chrome:

Uncaught RangeError: byte length of Int32Array should be a multiple of 4
    at new Int32Array (<anonymous>)
    at new Decrypter (e078b76d-d433-48ec-a7ba-90c3f96bd0d8:548)
    at DecrypterWorker.self.onmessage (e078b76d-d433-48ec-a7ba-90c3f96bd0d8:647)
Decrypter @ e078b76d-d433-48ec-a7ba-90c3f96bd0d8:548
DecrypterWorker.self.onmessage @ e078b76d-d433-48ec-a7ba-90c3f96bd0d8:647

Firefox:

RangeError: attempting to construct out-of-bounds TypedArray on ArrayBuffer 95ef2b78-e7e8-ef4b-8ff6-a0491701a31e:548:29
    Decrypter blob:https://videojs-http-streaming.netlify.app/95ef2b78-e7e8-ef4b-8ff6-a0491701a31e:548
    onmessage blob:https://videojs-http-streaming.netlify.app/95ef2b78-e7e8-ef4b-8ff6-a0491701a31e:647

Additional Information

versions

videojs

7.8.3

browsers

Chrome 86, Firefox 81

OSes

macOS 10.15.7

plugins

vhs 2.2.0

[gkatsev]

Unfortunately, SAMPLE-AES isn't supported right now.

[RafalLukawiecki]

Thank you. Let me retest with segment AES.

[RafalLukawiecki]

I am afraid it won't play segment AES either. Please have a look at this stream: https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8 using your hosted player.

In Chrome the error is:

Uncaught TypeError: Cannot read property 'video' of undefined
    at handleSegmentBytes (videojs-http-streaming.js:9497)
    at Worker.decryptionHandler (videojs-http-streaming.js:9650)
handleSegmentBytes @ videojs-http-streaming.js:9497
decryptionHandler @ videojs-http-streaming.js:9650

In Firefox the error is:

Uncaught TypeError: tracks is undefined

Is this a problem with my stream or VHS/video.js? I am happy to provide more info if that helps.

PS. It plays in Safari without any issues.

[gkatsev]

AES-128 should work. We test it on a regular basis https://videojs-http-streaming.netlify.app/?debug=false&autoplay=false&muted=false&minified=false&liveui=false&partial=false&url=https%3A%2F%2Fplayertest.longtailvideo.com%2Fadaptive%2Foceans_aes%2Foceans_aes.m3u8&type=application%2Fx-mpegurl&keysystems=&buffer-water=false However, it's definitely possible that there's some kind of bug there.

[RafalLukawiecki]

That does not look like fragmented MP4 but more like MPEG-TS. Are you testing AES with fMP4? Please have a look at the stream I shared (https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8) to see if it plays for you. Do you think the issue is with my stream or VHS/VJS?

If it would be helpful, I can encode a few of those problematic fMP4 test streams so that you can see the issues on an ongoing basis.

[RafalLukawiecki]

I have retested just now using video.js 7.10.1 and the built-in VHS 2.2.3 and I am afraid Segment-AES still does not work for fMP4 containers.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[RafalLukawiecki]

Still an issue with videojs 7.11.0 and vhs 2.4.1:

• Cannot play SEGMENT-AES: https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8
• Cannot play SAMPLE-AES: https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-sample-aes/m.m3u8

[video-archivist-bot]

Hey! We've detected some video files in a comment on this issue. If you'd like to permanently archive these videos and tie them to this project, a maintainer of the project can reply to this issue with the following commands:

• for https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-segment-aes/m.m3u8: say @video-archivist-bot save zEpzwd
• for https://d2gw5qqm1tusud.cloudfront.net/pml-test-fmp4-sample-aes/m.m3u8: say @video-archivist-bot save 9E3gE0

[gkatsev]

Unfortunately, we don't support SAMPLE-AES at all right now, and it does seem like we never got AES-128 working with fmp4 segments rather than TS segments.

[freddyamsterdam]

@RafalLukawiecki we were facing the same issue. After some trial and error, I figured out that the initiation vector seems to be the issue. I simply encoded our files with an empty string as an initiation vector and made sure to pass an empty IV to the playlist file:

#EXT-X-KEY:METHOD=AES-128,URI="pb-alpha-9cdffa16bbd5ccf0-0.key",IV=

Hope it helps.

Freddy

[RafalLukawiecki]

Interesting, @freddyamsterdam. I will try encoding a new video with an empty IV to see if that helps with our streams. In any case, if that works it would suggest there is a problem with how the IV value is read or used with fMP4 segments in video.js, which could be a good debugging hint. Thanks.

[freddyamsterdam]

@RafalLukawiecki it doesn't seem to work with multiple representation playlists. 😞 Will post an update if I manage to make any progress.

[gkatsev]

Oh, interesting that it works with an empty IV. It definitely works with ts segments, I wonder what's there that's failing for fmp4 segments.

[freddyamsterdam]

@gkatsev didn't mention before, but we're using TS segments 😢

[gkatsev]

@freddyamsterdam TS segments should work with IV properly. Did you find it didn't? If so, can you open a separate issue? And if you're able to provide a sample stream, that would be awesome.

[RafalLukawiecki]

Thank you, both, for following up. I am unsure if and what I can do to help troubleshoot it further. The test stream is still available.

[freddyamsterdam]

@gkatsev sure thing!

@RafalLukawiecki did the empty IV not work out for you?

[RafalLukawiecki]

So sorry, I have not rendered another test stream with an empty IV, yet. I will do that.

[iroha7941]

The issue with fMP4 appears to be that the init segments are not being decrypted in http-streaming. This also explains why TS works but fMP4 does not, since init segments are only used for fMP4 but not for TS.

The example2.zip reproducer in https://github.com/videojs/video.js/issues/7051 shows that encrypting the init segment works with the Apple native decoder. Also the HLS spec (RFC 8216) notes that the init segments are encrypted: https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-08#section-4.4.4.4

I started writing a patch but I suspect someone more familiar with the project might be able to fix this faster/better.

[RafalLukawiecki]

@iroha7941, unless someone else comes up with a solution, I would be happy to help test your patch.

[gkatsev]

@iroha7941 oh, that makes a lot of sense. We probably didn't think that the init segments could also be encrypted when we added fmp4 support, and we didn't have any test content back then. If you're able to provide a patch that would be amazing! We'll happily review and help out, feel free to ask questions in the #playback channel on our slack: http://slack.videojs.com

[iroha7941]

I created patches for http-streaming and m3u8-parser that enable video.js to playback the encrypted fMP4 files I've created. I created a repo containing a patched build of video.js and links to the branches in my forked repos which contain the patches: https://github.com/iroha7941/video.js-build

I have not yet done a lot of testing (and I haven't really tested for byte-range segments), and am still having problems running unit tests for http-streaming, but this seems to work for me.

[gkatsev]

@iroha7941 I finally got around to looking at the m3u8-parser PR and didn't see it working with my local VHS copy, then I noticed that you had VHS related changes as well. Is there a reason why you didn't make a PR for it? Is it because it's multiple commits behind main? I can confirm that your changes do work.

[iroha7941]

@gkatsev I hadn't created a PR for VHS as I still haven't been able to run testsuite successfully (many tests fail with "vhs is undefined"), let alone add appropriate tests. I'm guessing this is just a problem with my setup or something (I'm still new to Javascript/npm development), but I haven't had time to look into it more. When I have a patch with appropriate tests ready I'll sumbit a PR. Sorry for the delay.

[gkatsev]

If you make a PR we can try and help figure out what's wrong. Either way, thanks!

[index23]

Hi @gkatsev, did you get to play SAMPLE-AES encrypted fmp4 files with HLS?

[venomone]

@iroha7941 I have the same question. I encrypt every of my ABR tracks with a different SAMPLE-AES key and pass it to the m3u8 playlist of that specific track. Sadly, I also don't get VideoJS to play the Stream :( Would be awesome to finally see this feature