Error on loading SmoothStreaming + ClearKey encryption

[gabrielheming]

When trying to load a SmoothStreaming protocol using AES Clear Key as encryption I receive the following error:

Uncaught OtherError: OtherError (PIPELINE_PARSE_ERROR) Error: Cannot parse PlayReady private data: invalid XML

This also has been tested on the online test player: https://developers.canal-plus.com/rx-player/

It fails parsing the manifest XML before trying to get the key.

[peaBerberian]

Hi, From re-reading the code linked to the parsing of the Smooth Manifest, it seems that the Error is thrown when the Base64 data inside the <ProtectionHeader> element (inside a <Protection> element), translates to a PSSH that is assumed to be a PlayReady PSSH (and which translates into XML data) but does not contain the required <KID> element.

Can I see your Manifest?

[gabrielheming]

Sure, this is the manifest provided by Azure Media Services:

<?xml version="1.0" encoding="UTF-8"?>
<SmoothStreamingMedia MajorVersion="2" MinorVersion="2" Duration="10343666" TimeScale="10000000">
	<StreamIndex Chunks="1" Type="video" Url="QualityLevels({bitrate})/Fragments(video={start time})" QualityLevels="4">
		<QualityLevel Index="0" Bitrate="38811985" FourCC="H264" MaxWidth="3840" MaxHeight="2160" CodecPrivateData="0000000167640033ACD9403C0043EC05A8300832000007D20001D4C01E30632C0000000168EBECB22C"/>
		<QualityLevel Index="1" Bitrate="25390539" FourCC="H264" MaxWidth="2560" MaxHeight="1440" CodecPrivateData="0000000167640032ACD9402800B5B016A0C020C800001F480007530078C18CB00000000168EBECB22C"/>
		<QualityLevel Index="2" Bitrate="8315032" FourCC="H264" MaxWidth="1920" MaxHeight="1080" CodecPrivateData="0000000167640028ACD940780227E5C05A8300832000007D20001D4C01E30632C00000000168EBECB22C"/>
		<QualityLevel Index="3" Bitrate="3413652" FourCC="H264" MaxWidth="1280" MaxHeight="720" CodecPrivateData="0000000167640020ACD9405005BB016A0C020C800001F480007530078C18CB0000000168EBECB22C"/>
		<c t="0" d="10343666"/>
	</StreamIndex>
	<Protection>
		<ProtectionHeader SystemID="[REMOVED]">
			<ContentProtection schemeIdUri="urn:mpeg:dash:sea:2012" xmlns:sea="urn:mpeg:dash:schema:sea:2012">
				<sea:SegmentEncryption schemeIdUri="urn:mpeg:dash:sea:aes128-cbc:2013"/>
				<sea:KeySystem keySystemUri="urn:mpeg:dash:sea:keysys:http:2013"/>
				<sea:CryptoPeriod IV="[REMOVED]" keyUriTemplate="https://[REMOVED].media.azure.net/?kid=[REMOVED]"/>
			</ContentProtection>
		</ProtectionHeader>
	</Protection>
</SmoothStreamingMedia>

[peaBerberian]

OK thanks, it looks like the format is different than what we're used to.

It's difficult to find specifications/resources on Smooth Streaming and even more on how SmoothStreaming + ClearKey encryption is supposed to work (I admit that I'm not even sure how ClearKey is supposed to work here e.g. I can see the "keyUriTemplate" which gives me the impression of a license server to call, but what's the initialization data we have to provide first to a MediaKeySession?).

[gabrielheming]

Yes, you are right. I can send you the demo below from Azure Media Player demo that has the JWT token embedded:

https://ampdemo.azureedge.net/?url=%2F%2Famssamples.streaming.mediaservices.windows.net%2F830584f8-f0c8-4e41-968b-6538b9380aa5%2FTearsOfSteelTeaser.ism%2Fmanifest&aes=true&aestoken=Bearer%3DeyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1cm46bWljcm9zb2Z0OmF6dXJlOm1lZGlhc2VydmljZXM6Y29udGVudGtleWlkZW50aWZpZXIiOiI5ZGRhMGJjYy01NmZiLTQxNDMtOWQzMi0zYWI5Y2M2ZWE4MGIiLCJpc3MiOiJodHRwOi8vdGVzdGFjcy5jb20vIiwiYXVkIjoidXJuOnRlc3QiLCJleHAiOjE3MTA4MDczODl9.lJXm5hmkp5ArRIAHqVJGefW2bcTzd91iZphoKDwa6w8

The only thing that I see is that it request the CryptoPeriod URL using the provided token as authorization.

[peaBerberian]

OK thanks I looked at the URL and read a little more the specification regarding clearkey: https://www.w3.org/TR/encrypted-media/#clear-key

---

What surprised me in your URL, is that the media player doesn't seem to use EME (encrypted media extensions, the web APIs allowing to decrypt content). This makes me think that they are doing the decryption directly in JavaScript, which I guess should be possible in ClearKey (but only when clear key is used).

I guess that they have a custom logic doing a post at the URL found in that protection data, then they extract the key from the response, to then perform the decryption directly from JS (I may be wrong, yet I see no call to the main EME API and the video element does not seems to be linked to the mandatory MediaKeys instance).

---

Yet I don't know if what they're doing is what is "standardized" for smooth (and sadly standards on smooth streaming are hard to find!) or if this is a custom logic specially for this application.

From what I saw both from the EME specification and from the shaka-player for example, a player should at minimum know about the "key-id" of the wanted key, so it can perform the generateRequest API (allowing to request a license, using the EME APIs) or even both the key-id and key, so it can both perform generateRequest and the update call (so it can communicate the clearkey license through EME APIs). In the given page, the key-id is just somewhere in the license server's URL query string (and I guess the key is in the response), so hardly something that seems parsable by a player (as this way of doing it might not be portable for other contents).

But perhaps this is to be expected in smooth streaming contents? I have no idea :s

[gabrielheming]

I think it is something expected in smooth streaming as it is Microsoft's protocol. Also, the AMP is the Azure Media Service's proprietary player (though it not as good or flexible as yours).

What I guess that is happening it is that Azure Media Service platform handles the EME, because all configurations on encryption lies on Azure's side.

[gabrielheming]

I have changed to use DASH instead of Smoothing, and I am tried to test on your online demo player, I got another error:

Uncaught MediaError: MediaError (MEDIA_ERR_SRC_NOT_SUPPORTED) The media resource has been found to be unsuitable.

It also happens before issuing the license.

Below are my manifest (cmaf and csf):

<?xml version="1.0" encoding="utf-8"?>
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" xmlns:sea="urn:mpeg:dash:schema:sea:2012" mediaPresentationDuration="PT1.034S" minBufferTime="PT2S">
    <Period>
        <AdaptationSet id="1" group="1" profiles="ccff" bitstreamSwitching="false" segmentAlignment="true" contentType="video" mimeType="video/mp4" maxWidth="3840" maxHeight="2160" startWithSAP="1">
            <ContentProtection schemeIdUri="urn:mpeg:dash:sea:2012">
                <sea:SegmentEncryption schemeIdUri="urn:mpeg:dash:sea:aes128-cbc:2013"/>
                <sea:KeySystem keySystemUri="urn:mpeg:dash:sea:keysys:http:2013"/>
                <sea:CryptoPeriod keyUriTemplate="https://[REMOVED].westeurope.media.azure.net/?kid=[REMOVED]" IV="[REMOVED]"/>
            </ContentProtection>
            <SegmentTemplate timescale="10000000" media="QualityLevels($Bandwidth$)/Fragments(video=$Time$,format=mpd-time-cmaf,encryption=cbc)" initialization="QualityLevels($Bandwidth$)/Fragments(video=i,format=mpd-time-cmaf,encryption=cbc)">
                <SegmentTimeline>
                    <S d="10343666"/>
                </SegmentTimeline>
            </SegmentTemplate>
            <Representation id="1_V_video_1" bandwidth="38811985" codecs="avc1.640033" width="3840" height="2160"/>
            <Representation id="1_V_video_2" bandwidth="25390539" codecs="avc1.640032" width="2560" height="1440"/>
            <Representation id="1_V_video_3" bandwidth="8315032" codecs="avc1.640028" width="1920" height="1080"/>
            <Representation id="1_V_video_4" bandwidth="3413652" codecs="avc1.640020" width="1280" height="720"/>
        </AdaptationSet>
    </Period>
</MPD>

<?xml version="1.0" encoding="utf-8"?>
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" xmlns:sea="urn:mpeg:dash:schema:sea:2012" mediaPresentationDuration="PT1.034S" minBufferTime="PT2S">
	<Period>
		<AdaptationSet id="1" group="1" profiles="ccff" bitstreamSwitching="false" segmentAlignment="true" contentType="video" mimeType="video/mp4" codecs="avc1.640033" maxWidth="3840" maxHeight="2160" startWithSAP="1">
			<ContentProtection schemeIdUri="urn:mpeg:dash:sea:2012">
				<sea:SegmentEncryption schemeIdUri="urn:mpeg:dash:sea:aes128-cbc:2013"/>
				<sea:KeySystem keySystemUri="urn:mpeg:dash:sea:keysys:http:2013"/>
				<sea:CryptoPeriod keyUriTemplate="https://[REMOVED].westeurope.media.azure.net/?kid=[REMOVED]" IV="[REMOVED]"/>
			</ContentProtection>
			<SegmentTemplate timescale="10000000" media="QualityLevels($Bandwidth$)/Fragments(video=$Time$,format=mpd-time-csf)" initialization="QualityLevels($Bandwidth$)/Fragments(video=i,format=mpd-time-csf)">
				<SegmentTimeline>
					<S d="10343666"/>
				</SegmentTimeline>
			</SegmentTemplate>
			<Representation id="1_V_video_1" bandwidth="38811985" width="3840" height="2160"/>
			<Representation id="1_V_video_2" bandwidth="25390539" codecs="avc1.640032" width="2560" height="1440"/>
			<Representation id="1_V_video_3" bandwidth="8315032" codecs="avc1.640028" width="1920" height="1080"/>
			<Representation id="1_V_video_4" bandwidth="3413652" codecs="avc1.640020" width="1280" height="720"/>
		</AdaptationSet>
	</Period>
</MPD>

[peaBerberian]

Thank you for your DASH example, it helps a lot as I'm more familiar with it and above all there's a lot more resources on it accessible.

From what I can see, it seems that both the Smooth and DASH issues are linked to what seems to be Azure Media Services specificities which are poorly handled by the RxPlayer:

• First ContentProtection elements which usually helps us with obtaining decryption keys are in a unusual format following a "urn:mpeg:dash:sea:2012" scheme. After doing some reading, it seems to be described by this standard: https://www.iso.org/standard/73603.html (which I do not have access to).

  Players more usually encounter the urn:mpeg:dash:mp4protection:2011 scheme and/or key-system-specific (like Widevine, PlayReady or even ClearKey) urn:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx schemes which are all described by the DASH-IF IOP, a standard usually more closely followed by DASH players. This is also what most other DASH packagers are doing.

• It seems to necessitate that the decryption happen through the JavaScript application (for example by using WebCrypto APIs) instead of the more usual EncryptedMediaExtensions.

  I looked at how players were handling this and it seems that due to the complexity and its unusual-ness, support is relatively poor. From what I saw, dash.js and the shaka-player does not seem to support it either for now (though shaka-player welcomed external contributions on it). Video.js seems to be better but appear to still have some issues with it.

---

From there, I fear that implementing its support might be very time-consuming. That added with the fact that it's not a priority for us at Canal + for now mean that there's few chance that we will work on this in the near future :/

Still, if you really need it, your contributions are very welcomed.

Also, I'm under the impression that it's only their implementation of ClearKey which suffers from these issues. If you're able to use other technologies such as PlayReady and Widevine, support might be much better.

[gabrielheming]

Unfortunatelly, I have to use ClearKey to allow screenshots which I can't do with more sofisticated DRM. Also, we allow downloading the original video, but this wouldn't be a problem.

Because my schedule of implementation is short on my project, I will check if I can help you implementing it.