Persistent Widevine L3 Playback Failures: OEMCrypto_DecryptCENC failed: status = 36

[KanghyunJeon-CJ]

Version

Media3 1.6.1

More version details

Hello, We are encountering a critical and persistent issue affecting Widevine DRM playback in our Android application. This happens with many kinds of AndroidTV devices. The problem is 100% reproducible once it starts, and it persists even after standard troubleshooting steps like device reboots and app reinstalls.

The core exception reported by MediaCodec is consistently: Caused by android.media.MediaCodec$CryptoException: Crypto key not available: 1 (Unknown error -1)

Here's a representative snippet from our crash logs.

6006 ERROR_CODE_DRM_SYSTEM_ERROR
Caused by androidx.media3.exoplayer.s

MediaCodecAudioRenderer error, index=1, format=Format(18, null, audio/mp4, audio/mp4a-latm, mp4a.40.2, 129852, ko, [-1, -1, -1.0, null], [2, 48000]), format_supported=YES

Caused by android.media.MediaCodec$CryptoException
Crypto key not available: 1 (Unknown error -1)
============================== Beginning of DRM Plugin Log ==============================
07-15 12:57:33.971 I found instance=clearkey [email protected]::IDrmFactory
07-15 12:57:33.971 I found instance=playready [email protected]::IDrmFactory
07-15 12:57:33.971 I found instance=widevine [email protected]::IDrmFactory
07-15 12:57:34.709 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.724 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.725 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.730 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.732 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.738 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.740 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.741 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.741 I [cdm_engine.cpp(190):OpenSession] New session: session_id = sid359
07-15 12:57:34.741 I [cdm_engine.cpp(929):QueryOemCryptoSessionId] session_id = sid359
07-15 12:57:34.759 I [cdm_engine.cpp(2044):SetPlaybackId] session_id = sid359, playback_id = uiAE_09h19I1MlmA
07-15 12:57:34.771 I [cdm_engine.cpp(277):GenerateKeyRequest] session_id = sid359, key_set_id = <empty>, license_type = Streaming
07-15 12:57:34.771 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.771 I [file_utils.cpp(40):Exists] stat failed: ENOENT
07-15 12:57:34.982 I [cdm_engine.cpp(834):QuerySessionStatus] session_id = sid359
07-15 12:57:35.222 I [cdm_engine.cpp(834):QuerySessionStatus] session_id = sid359
07-15 12:57:40.212 I [cdm_engine.cpp(356):AddKey] session_id = sid359, key_set_id = <empty>
07-15 12:57:40.340 E [crypto_session.cpp(1692):Decrypt] OEMCrypto_DecryptCENC failed: status = 36
============================== End of DRM Plugin Log ==============================

Can you guys suggest solutions or wokraround? we would be grateful for any suggestions or guidance on potential troubleshooting steps or temporary workarounds that we could provide to our users.

Thank you!

Devices that reproduce the issue

• Chromecast Google TV
• U+tv UHD3

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

1. play drm contents
2. video start

Expected result

video start playing

Actual result

video start failure with 6006 drm error

Media

a few users with any widevine contents

Bug Report

• [ ] You will email the zip file produced by adb bugreport to [email protected] after filing this issue.

[icbaker]

Please reproduce the issue, run adb bugreport and email the zip file produced to [email protected] with subject Issue #2605 - then update this issue when you've done this.

[KanghyunJeon-CJ]

We fully understand the importance of adb bugreports for deep-level diagnosis. However, as this issue is not reproducible on our internal development devices and appears to be specific to certain user hardware/software configurations, obtaining these comprehensive logs directly from the affected end-users is a big challenges. Could you possibly give us some ideas based on the crash log we wrote above?

[icbaker]

@rrfrias PTAL and see if you can determine anything from the logcat snippet above.

[rrfrias]

That error indicates that the license key expired. If users are experiencing this occasionally, it might be a little hard to identify the specific error. Do the errors occurs randomly though the day or can you associate them with times of day/content/devices/device releases/users etc? it is a bit surprising because the expiration occurs on license delivery.

If you are able to capture a license issued to a device that reports this error, we can help analyze. Please email [email protected] and mention "Issue 2605" in the subject line.

If you are a Widevine Licensee, you can reach out to https://www.widevine.com/contact and we can share more information that detail how to set license policies.

The app can also register for events ExoMediaDrm::OnExpirationUpdateListener and ExoMediaDrm::OnKeyStatusChangeListener . The latter will allow you to be notified when keys are available (or not) for use.

[KanghyunJeon-CJ]

For users experiencing this issue, once it occurs, the same error persists for more than a day, even after reinstalling the app. This is occurring on many kinds of devices.

As you suggested, we will provide you with the logs using the code snippet below.

mediaDrm.setOnKeyStatusChangeListener { exoMediaDrm, bytes, keyStatuses, hasNewUsableKey ->
                            val drmSessionId = Base64.encodeToString(bytes, Base64.NO_WRAP)
                            PlayerLog.e(TAG, "DRM onKeyStatusChange: sessionId = $drmSessionId / hasNewUsableKey = $hasNewUsableKey")

                            keyStatuses.forEach { keyStatus ->
                                val keyId = Base64.encodeToString(keyStatus.keyId, Base64.NO_WRAP)
                                val statusCode = keyStatus.statusCode
                                val securityLevel = exoMediaDrm.getPropertyString("securityLevel")
                                val version = exoMediaDrm.getPropertyString("version")
                                val vendor = exoMediaDrm.getPropertyString("vendor")
                                PlayerLog.e(TAG, " -> KeyId: $keyId, " +
                                        "Status Code: $statusCode securityLevel: $securityLevel, version: $version, vendor: $vendor")
                            }
                        }

mediaDrm.setOnExpirationUpdateListener { exoMediaDrm, bytes, expirationTimeMs ->
                            val drmSessionId = Base64.encodeToString(bytes, Base64.NO_WRAP)
                            PlayerLog.e(TAG, "DRM onExpirationUpdate -> drmSessionId: $drmSessionId / expirationTimeMs = $expirationTimeMs")
                            val securityLevel = exoMediaDrm.getPropertyString("securityLevel")
                            val version = exoMediaDrm.getPropertyString("version")
                            val vendor = exoMediaDrm.getPropertyString("vendor")
                            PlayerLog.e(TAG, " -> securityLevel: $securityLevel, version: $version, vendor: $vendor")
                        }

[KanghyunJeon-CJ]

I just sent you guys an email for the device log.

In our problematic application version, we had configured the DRM system to be fixed at Widevine L3. However, in our latest application version, we have switched our DRM configuration back to the default security level. Since making this change, we dont get the OEMCrypto_DecryptCENC failed: status = 36 error anymore.  The issue seems to be probably resolved. We still have another app that use only L3 too. So, we would greatly appreciate your analysis and an explanation of why forcing the L3 security level triggered this specific status=36 

Thank you.

[rrfrias]

Thanks for the log. This was run on android 13. Might it be possible to get a log for a device running a later version?

[KanghyunJeon-CJ]

Im afraid ican't. This is the only log we have, and it was very difficult to obtain. There are devices which has problem on later Android version too but majority of cases reported on Android 13.

[KanghyunJeon-CJ]

@rrfrias I just sent you guys an email for the later version(Android 15) log.

[rrfrias]

Thanks for that bugreport. While the device is running Android 15 the DRM vendor partition is running an older version (released with android 12 or 13) so we are not seeing the logs that we were hoping for. Is it possible to capture the second parameter response passed to MediaDrm#provideKeyResponse (byte[] scope, byte[] response) and share with us, when an error occurs?

Soon after the license is processed (150 ms), a decryption request comes in and the key is noted as expired. You might want to check the duration values that are being set by the license service.

For users experiencing this issue, once it occurs, the same error persists for more than a day, even after reinstalling the app. This is occurring on many kinds of devices.

It is surprising that it persists across sessions and reinstalls. I assume they have tried a reboot as well. Does it resolve itself after a day or is the device stuck? Are these streaming licenses or for offline/downloaded scenarios?

[KanghyunJeon-CJ]

Thank you for your detailed analysis.

1. In response to your request, call to provideKeyResponse is managed deep within the media framework and is not directly exposed at our application layer. Could you please advise on the recommended procedure for capturing this specific response byte array?

2. We have also reviewed the duration values being set by our license service. We found our license duration was set to 7 days.

3. To answer your other questions:

• The error, once it occurs, does not resolve itself after a day. The affected devices appear to be stuck with the error until we set the DRM security Default from L3.
• These are streaming licenses, not for offline/downloaded scenarios.

We hope this information helps in your investigation.

[icbaker]

1. In response to your request, call to provideKeyResponse is managed deep within the media framework and is not directly exposed at our application layer. Could you please advise on the recommended procedure for capturing this specific response byte array?

You can create a wrapping instance of MediaDrmCallback and provide it when building your DefaultDrmSessionManager.

It should look something like this:

public final class KeyResponseLoggingDrmCallback implements MediaDrmCallback {

  private final MediaDrmCallback delegate;

  public KeyResponseLoggingDrmCallback(MediaDrmCallback delegate) {
    this.delegate = delegate;
  }

  @Override
  public final byte[] executeKeyRequest(UUID uuid, ExoMediaDrm.KeyRequest request) {
    byte[] keyResponse = delegate.executeKeyRequest(uuid, request);
    Log.w("2605", "keyResponse=0x" + Util.toHexString(keyResponse));
    return keyResponse;
  }

  @Override
  public final byte[] executeProvisionRequest(UUID uuid, ExoMediaDrm.ProvisionRequest request) {
    delegate.executeProvisionRequest(uuid, request);
  }
}

You probably want to use HttpMediaDrmCallback as your delegate.

[rrfrias]

2. We have also reviewed the duration values being set by our license service. We found our license duration was set to 7 days.

Is rental or playback duration being set?

3. To answer your other questions: The error, once it occurs, does not resolve itself after a day. The affected devices appear to be stuck with the error until we set the DRM security Default from L3. These are streaming licenses, not for offline/downloaded scenarios.

It is surprising that this persists. Streaming sessions should not carry over state. It would seem more plausible if the failure was tied to the device or license. I assume you have verified that a title that was playback before the failure is no longer playable.

Can you capture and share the license using the snippet from the previous comment?

[KanghyunJeon-CJ]

@icbaker Thanks for the code snippet.

@rrfrias Just make sure that Failure Occurs on L3, Works on L1 (Default). We just set expire_date for 7 days and set those rental or playback duration 0. Do we need to set larger number than 0? I don't know if Doverunner set something else after our request.

Anyway, i'll get back to you guys with the logs.

[KanghyunJeon-CJ]

I just checked again for those durations. stream = null download = 0

[rrfrias]

Widevine has some guidelines on how to set policies. If you are a Widevine licensee/partner you can contact https://widevine.com/contact for more information/documentation.

Your policies are slightly out of line with recommendations but should not cause the immediate expiration that we are seeing.

Next steps

1. Can you capture a license/key Response when the error occurs as mentioned in https://github.com/androidx/media/issues/2605#issuecomment-3199944159
2. A test version of your app of your app might be helpful to see if we can repro. Have you tried putting in a loop license acquisition+playback to see if the error reproduced? Please email [email protected] and mention "Issue 2605" in the subject line.

The error, once it occurs, does not resolve itself after a day. The affected devices appear to be stuck with the error until we set the DRM security Default from L3.

Is this based on telemetry data or from customer reports/feedback. I am trying to understand when this error occurs whether this affects all media apps or only your app?

[KanghyunJeon-CJ]

Thanks for the response. We are looking at those durations.

Next steps

1. Can you capture a license/key Response when the error occurs as mentioned in https://github.com/androidx/media/issues/2605#issuecomment-3199944159

Yes, we got the firebase log for that and I'll send it to you via email. Email Title: "[ Issue 2605 ] a capture of license/key Response"

2. A test version of your app of your app might be helpful to see if we can repro. Have you tried putting in a loop license acquisition+playback to see if the error reproduced? Please email [email protected] and mention "Issue 2605" in the subject line.

We currently only serve our service in Korea, so we apologize for the inconvenience. No, we haven't tried a loop license acquisition + playback. For that test, what exactly should we do?

3. Is this based on telemetry data or from customer reports/feedback. I am trying to understand when this error occurs whether this affects all media apps or only your app?

Initially, we became aware of this issue through user reports, and we could also see continuous video playback errors due to DRM errors in our logs.

This is more error callback which is slightly different.

Caused by android.media.MediaCodec$CryptoException: Error decrypting data: requested key has not been loaded: ERROR_DRM_NO_LICENSE
cdm err: 5, oem err: 36, ctx: 23
============================== Beginning of DRM Plugin Log ==============================
  09-03 10:53:18.339 I No hidl drm factories found
  09-03 10:53:18.340 W Failed to find passthrough drm factories
  09-03 10:53:19.577 I No hidl drm factories found
  09-03 10:53:19.577 W Failed to find passthrough drm factories
  09-03 10:54:53.065 I No hidl drm factories found
  09-03 10:54:53.065 W Failed to find passthrough drm factories
  09-03 10:54:56.604 I No hidl drm factories found
  09-03 10:54:56.604 W Failed to find passthrough drm factories
  09-03 10:54:59.068 I No hidl drm factories found
  09-03 10:54:59.068 W Failed to find passthrough drm factories
  09-03 10:55:02.398 I No hidl drm factories found
  09-03 10:55:02.398 W Failed to find passthrough drm factories
  09-03 10:55:02.933 I [cdm_engine.cpp(978):QuerySessionStatus] session_id = sid342
  09-03 10:55:02.960 I [cdm_engine.cpp(384):AddKey] session_id = sid342, key_set_id = <empty>
  09-03 10:55:03.034 I [cdm_engine.cpp(1112):IsSecurityLevelSupported] level = L1
  09-03 10:55:03.034 E [cdm_session.cpp(732):Decrypt] Decryption failed: sid = sid342, status = 5
  09-03 10:55:03.034 E [crypto_session.cpp(2011):Decrypt] OEMCrypto_DecryptCENC failed: oec_session_id = 1000, security_level = Default, status = 36
  09-03 10:55:03.074 I [cdm_engine.cpp(978):QuerySessionStatus] session_id = sid342
  09-03 10:55:03.182 E [cdm_session.cpp(732):Decrypt] Decryption failed: sid = sid342, status = 5
  09-03 10:55:03.323 I [cdm_engine.cpp(263):CloseSession] session_id = sid342
  09-03 10:55:03.332 I [(0):] L3 Terminate.
  09-03 10:55:03.332 I [oemcrypto_adapter_dynamic.cpp(888):Level1Terminate] L1 Terminate
  09-03 10:55:13.346 W Failed to find passthrough drm factories
  09-03 10:55:13.346 I No hidl drm factories found
  09-03 10:55:13.348 I [(0):] Level3 Library 33098 May 14 2024 19:04:52
  09-03 10:55:13.348 I [cdm_engine.cpp(1112):IsSecurityLevelSupported<truncated: 9198 chars>
       at android.media.MediaCodec.native_queueSecureInputBuffer(MediaCodec.java)
       at android.media.MediaCodec.queueSecureInputBuffer(MediaCodec.java:3293)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doQueueSecureInputBuffer(AsynchronousMediaCodecBufferEnqueuer.java:233)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doHandleMessage(AsynchronousMediaCodecBufferEnqueuer.java:194)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.access$000(AsynchronousMediaCodecBufferEnqueuer.java)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer$1.handleMessage(AsynchronousMediaCodecBufferEnqueuer.java:91)
       at android.os.Handler.dispatchMessage(Handler.java:107)
       at android.os.Looper.loopOnce(Looper.java:257)
       at android.os.Looper.loop(Looper.java:342)
       at android.os.HandlerThread.run(HandlerThread.java:85)
        

[rrfrias]

Yes, we got the firebase log for that and I'll send it to you via email. Email Title: "[ Issue 2605 ] a capture of license/key Response"

Thanks. There are 6 files with 1-4 key responses per file. Are all of those associated with a decryption failure?

No, we haven't tried a loop license acquisition + playback. For that test, what exactly should we do?

I was wondering if you might try to reproduce the problem locally. Automate this test:

• pick a random title
• playback some content (acquire license etc)
• if playback is successful and continues for a period of time (lets say 10 seconds), repeat steps with another title.
• Continue until you encounter a failure

This is more error callback which is slightly different.

In this log and the log from the first comment, the key expired soon after being loaded. MediaCodec.CryptoExceptions were thrown in both cases. In this case above there might have been a follow on CryptoException because of data queued in the buffer. It is a bit hard to say without access to the logcat.

Internal tracker: 443075022

[rrfrias]

Some more questions,

1. Can you share a stack trace from Chromecast Google TV?
2. Are there any other devices experiencing the problem
3. Can you provide the android OS version for the devices experiencing the problem and also what % of devices are experiencing the problem?

Just to confirm from the previous comment

Thanks. There are 6 files with 1-4 key responses per file. Are all of those associated with a decryption failure?

[KanghyunJeon-CJ]

Thanks. There are 6 files with 1-4 key responses per file. Are all of those associated with a decryption failure?

Yes. They are all associated with error 36.

I was wondering if you might try to reproduce the problem locally. Automate this test:

We haven't been able to reproduce the issue locally. We suspect it may be difficult, but we will continue to try.

1. Can you share a stack trace from Chromecast Google TV?

On the Chromecast device, we didn't get the error 36. Instead, we received an ERROR_DRM_CANNOT_HANDLE.

Caused by android.media.MediaCodec$CryptoException: Operation not supported in this configuration: ERROR_DRM_CANNOT_HANDLE
cdm err: 0, oem err: 0, ctx: 0
============================== Beginning of DRM Plugin Log ==============================
  09-05 09:35:43.684 I found instance=castkey [email protected]::IDrmFactory
  09-05 09:35:43.684 I found instance=playready [email protected]::IDrmFactory
  09-05 23:20:39.038 I [file_utils.cpp(38):Exists] stat failed: ENOENT
  09-05 23:20:39.097 I [cdm_engine.cpp(1089):IsSecurityLevelSupported] level = L1
  09-05 23:20:39.099 I [cdm_engine.cpp(955):QuerySessionStatus] session_id = sid1898
  09-05 23:20:39.287 I [cdm_engine.cpp(382):AddKey] session_id = sid1898, key_set_id = <empty>
  09-05 23:20:39.325 I [cdm_engine.cpp(1089):IsSecurityLevelSupported] level = L1
  09-05 23:20:39.336 I [cdm_engine.cpp(955):QuerySessionStatus] session_id = sid1898
  09-05 23:50:05.831 I [cdm_engine.cpp(261):CloseSession] session_id = sid1898
  09-05 23:50:05.879 I [(0):] L3 Terminate.
  09-05 23:50:05.879 I [oemcrypto_adapter_dynamic.cpp(848):Level1Terminate] L1 Terminate
  09-06 00:10:42.409 I [file_utils.cpp(38):Exists] stat failed: ENOENT
  09-06 00:10:42.410 I [oemcrypto_adapter_dynamic.cpp(787):Initialize] Level 3 Build Info (v18): {"soc_vendor":"L3_34136","soc_model":"ARM 32 bit","ta_ver":"18.1.0+May 15 2024_23:17:25_","uses_opk":false,"tee_os":"none","tee_os_ver":"0.0.0","form_factor":"L3","implementer":"Widevine","fused":false}
  09-06 00:10:42.419 I [(0):] Level3 Library 34136 May 15 2024 23:17:25
  09-06 00:10:42.475 I [oemcrypto_adapter_dynamic.cpp(801):Initialize] L3 Initialized. Trying L1.
  09-06 00:10:42.580 I [oemcrypto_adapter_dynamic.cpp(835):Initialize] Level 1 Build Info (v18): {"soc_vendor":"Amlogic","soc_model":"T962X2","provision":"Provision 2.0 Keybox","ta_ver":"18.7.0+Jan 14 202514:51:28","uses_opk":true,"tee_os":"OP-TEE","tee_os_ver":"V2.4.4","form_factor":"mbx","implementer":"Widevine","fused":true,"is_debug":false}
  09-06 00:10:42.582 I [file_utils.cpp(38):Exists] stat failed: ENOENT<truncated: 8702 chars>
       at android.media.MediaCodec.native_queueSecureInputBuffer(MediaCodec.java)
       at android.media.MediaCodec.queueSecureInputBuffer(MediaCodec.java:3105)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doQueueSecureInputBuffer(AsynchronousMediaCodecBufferEnqueuer.java:233)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.doHandleMessage(AsynchronousMediaCodecBufferEnqueuer.java:194)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer.access$000(AsynchronousMediaCodecBufferEnqueuer.java)
       at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecBufferEnqueuer$1.handleMessage(AsynchronousMediaCodecBufferEnqueuer.java:91)
       at android.os.Handler.dispatchMessage(Handler.java:106)
       at android.os.Looper.loopOnce(Looper.java:205)
       at android.os.Looper.loop(Looper.java:294)
       at android.os.HandlerThread.run(HandlerThread.java:67)

I will send you DRM logs via email, TITLE : [ Issue 2605 ] a capture of license/key Response for ChromeCast.

2. Are there any other devices experiencing the problem

Yes, the issue occurs on a variety of devices.

3. Can you provide the android OS version for the devices experiencing the problem and also what % of devices are experiencing the problem?

Since "DRM system error 6006" can be triggered by issues other than "error 36", we cannot isolate the specific percentage for "error 36" alone. Anyway, The total 6006 we got is 0.2% of the total Android Users. These are the OS version of those user who face to it.

[KanghyunJeon-CJ]

Hi, guys We haven't heard about the issue. Any update or news will be very nice. Thanks.

[rrfrias]

Since we are unable to reproduce, we cannot root cause the problem. We have a few ideas of what might be causing it and are discussing ways to reduce the frequency of users encountering this issue. Keep in mind that when we do have a fix that we make available to device makers, they are not required to pick up this fix.