rx-player icon indicating copy to clipboard operation
rx-player copied to clipboard
verified publisher icon canalplus

BUFFER_APPEND_ERROR on Tizen with multiple codecs (AVC & HEVC)

Open el-gringo opened this issue 4 years ago • 7 comments

Hi RxPlayer team !

I have streams with mixed avc & hevc codecs with widevine protection.

The player try to switch between avc to hevc (or vice versa) but the video streams stop with the error Stream: video Stream crashed. Stopping playback. t: MediaError (BUFFER_APPEND_ERROR) Error: An unknown error occured when doing operations on the SourceBuffer

Is there a way to avoid switching between codecs ? I have tried transportOptions.representationFilter but at this stage I don't have an overview of all representations to be able to avoid one or other codec. I have tried keySystems { onCodecSwitch: 'reload' }, no change

el-gringo avatar Jun 17 '21 18:06 el-gringo

Hello @el-gringo :)

We already encountered a similar error on few devices like PS5 where the device can't handle the fact to push video segments of a new codec into the same SourceBuffer but with a different codec.

First, as you said, you can use the representationFilter.

function representationFilter(representation, infos) {
  if (infos.bufferType === "video") {
    const { codec } = representation;
    // If we encounter a representation with a HEVC codec, we will discard that representation, 
    // so we make sure to never play it.
    if (codec.indexOf('hvc') !== -1) {
       return false;
    }
    return true;
  }

  // Otherwise, allow all non-video representations
  return true;
}

You can see all the arguments passed to this function: https://github.com/canalplus/rx-player/blob/eb8e3aa9346de0a12c58de0f22d32b1ed50fcd75/src/manifest/adaptation.ts#L182

However, for your current use case, this is not recommended as you prevent urself from playing those contents on devices that support HEVC codec.

On the other hand, you can use the API (the recommended one for your use case and the one u tried):

Make sure that property is set at the right place, she doesn't belong to the keySystems API but rather at the root of the loadVideoOptions

You can have more information about this property on the release note v3.23.0, on the Codec switching behavior part.

Make sure to tell us how it went for you.

PaulRosset avatar Jun 18 '21 09:06 PaulRosset

Thanks for your fast answer, and my bad about the mistake with onCodecSwitch. Unfortunately I still have the BUFFER_APPEND_ERROR after fixing the loadVideo config. Anyway, I would avoid the reload strategy which might be annoying if the switching happens too often.

Here are the logs:

EME: Clearing-up EME session.
EME: Nothing to clear. Returning right away. No state = false
API: Calling loadvideo {autoPlay: true, defaultAudioTrack: undefined, defaultTextTrack: undefined, enableFastSwitching: true, hideNativeSubtitle: false, …}
EME: Clearing-up EME session.
EME: Nothing to clear. Returning right away. No state = false
API: playerStateChange event LOADING
EME: Searching for compatible MediaKeySystemAccess
EME: Found cached compatible keySystem {keySystemOptions: {…}, keySystemAccess: MediaKeySystemAccess}
Init: Creating MediaSource
Init: Attaching MediaSource URL to the media element blob:file:///4e9fff22-69b2-4e76-8ad8-c23e1c4dfe24
DASH Parser: merging "switchable" AdaptationSets 1 0
DASH Parser: merging "switchable" AdaptationSets 2 0
DASH Parser: merging "switchable" AdaptationSets 3 0
MF: Manifest parsed in 50.69000000000051ms
EME: The MediaKeys already has a server certificate, skipping...
Init: clock offset found for a live content, checking if we can start close to it
SO: Creating new Stream for video e {parsingErrors: Array(0), id: "0", adaptations: {…}, duration: undefined, start: 0, …}
SO: Creating new Stream for audio e {parsingErrors: Array(0), id: "0", adaptations: {…}, duration: undefined, start: 0, …}
Init: Updating duration 4294967296
Init: Resume playback speed 1
Stream: Updating video adaptation e {parsingErrors: Array(0), id: "0", isTrickModeTrack: false, type: "video", isClosedCaption: false, …} e {parsingErrors: Array(0), id: "0", adaptations: {…}, duration: undefined, start: 0, …}
SB: Adding native SegmentBuffer with codec video/mp4;codecs="avc1.64000d"
Stream: Updating audio adaptation e {parsingErrors: Array(0), id: "376", isTrickModeTrack: false, type: "audio", language: "fra", …} e {parsingErrors: Array(0), id: "0", adaptations: {…}, duration: undefined, start: 0, …}
SB: Adding native SegmentBuffer with codec audio/mp4;codecs="mp4a.40.2"
ABR: enter starvation mode.
Stream: changing representation video e {id: "382", bitrate: 3000000, codec: "avc1.64001f", height: 720, width: 1280, …}
EME: Creating a new temporary session
EME: Binding session events MediaKeySession {sessionId: "", expiration: NaN, closed: Promise, keyStatuses: MediaKeyStatusMap, onkeystatuseschange: null, …}
Compat: Trying to move CENC PSSH from init data at the end of it.
Stream: New active period e {parsingErrors: Array(0), id: "0", adaptations: {…}, duration: undefined, start: 0, …}
APP RxPlayer onPeriodChange e {parsingErrors: Array(0), id: "0", adaptations: {…}, duration: undefined, start: 0, …}
APP RxPlayer onVideoTracksChange [{…}]
APP RxPlayer onVideoTrackChange {id: "0", representations: Array(7)}
APP RxPlayer onVideoBitrateChange 3000000
Stream: changing representation audio e {id: "376", bitrate: 64000, codec: "mp4a.40.2", mimeType: "audio/mp4", contentProtections: {…}, …}
EME: Received message event, type license-request MediaKeySession {sessionId: "ksidB1344E0D", expiration: NaN, closed: Promise, keyStatuses: MediaKeyStatusMap, onkeystatuseschange: null, …} MediaKeyMessageEvent {isTrusted: true, messageType: "license-request", message: ArrayBuffer(1946), type: "message", target: MediaKeySession, …}
Init: Set initial time 1624011704.439635
Init: Pause playback to build buffer
EME: Updating MediaKeySession with message
Stream: slow Representation switch video
EME: MediaKeySession update succeeded.
EME: keystatuseschange event received MediaKeySession {sessionId: "ksidB1344E0D", expiration: 1624009915000, closed: Promise, keyStatuses: MediaKeyStatusMap, onkeystatuseschange: null, …} Event {isTrusted: true, type: "keystatuseschange", target: MediaKeySession, currentTarget: MediaKeySession, eventPhase: 2, …}
Init: Resume playback speed 1
Init: Can begin to play content
API: playerStateChange event LOADED
API: playerStateChange event PLAYING
APP RxPlayer onVideoBitrateChange 14800000
Stream: changing representation video e {id: "3519", bitrate: 14800000, codec: "hvc1.1.2.L123", height: 1080, width: 1920, …}
DASH Parser: merging "switchable" AdaptationSets 1 0
DASH Parser: merging "switchable" AdaptationSets 2 0
DASH Parser: merging "switchable" AdaptationSets 3 0
MF: Manifest parsed in 79.59500000000116ms
Stream: video Stream crashed. Stopping playback. t: MediaError (BUFFER_APPEND_ERROR) Error: An unknown error occured when doing operations on the SourceBuffer

el-gringo avatar Jun 18 '21 10:06 el-gringo

I have pushed a pull request to allow me more filtering possibilities. https://github.com/canalplus/rx-player/pull/971

However I am thinking about a more elegant way of doing this. Maybe adding a representationPrefilter option ?

el-gringo avatar Jun 18 '21 11:06 el-gringo

Thanks for the logs, it will help us.

So, I see multiple things out here:

  • You are using the adaptation-set-switching option from the DASHIF that allow us to merge multiple tracks into a single one. So, here we merge multiple AdaptationSet into a single one.
  • By using this, You will be limited as you will have to reload each time the adaptive bitrate manager will have to go from an AVC representation to an HEVC representation in case you choose to reload.
    If u choose the representationFilter, it will keep you away from playing the 1080P (HEVC) Content as you will discard any HEVC representation.
  • Also, by reading the logs, we don't see any RELOADING state, which normally should happen when you switch from the 720P quality to the 1080p as both codecs are not compatible. We only see:
API: playerStateChange event LOADED
API: playerStateChange event PLAYING
  • Do the Tizen TV support the HEVC codec? What happens if you only use the HEVC representation by forcing the video bitrate setVideoBitrate(14800000)?
  • Finally, we will discuss more in-depth about a potential new API with the team, to be able to address that kind of behavior we encounter not so often.

We will keep you in touch about what we plan to do in a near future and thanks you for your proposal, we will study it soon!

PaulRosset avatar Jun 18 '21 14:06 PaulRosset

  • I have to check with the one who create streams about adaptation-set-switching to avoid mixing AVC & HEVC in the same AdaptationSet
  • actually my representationFilter is to prefer HEVC and discard AVC
  • Yes I need to investigate why it does not reload
  • The TV supports HEVC and was working using the trick about forcing the bitrate before patching RxPlayer
  • Thanks for your time. For now the patch I pushed totally fit my needs. I know SmartTVs are a bit exotic ;-)

el-gringo avatar Jun 18 '21 14:06 el-gringo

I have to check with the one who create streams about adaptation-set-switching to avoid mixing AVC & HEVC in the same AdaptationSet

The fact that the RxPlayer merge all of those DASH AdaptationSet into one RxPlayer Adaptation is purely due to our implementation. What really is done by adding the supplemental adaptation-set-switching property is telling players that they can technically switch between all those AdaptationSet (e.g. with the player's adaptive streaming logic).

The DASH-IF envisages your case (multiple codecs in multiple AdaptationSet all linked through an adaptation-set-switching supplemental property), so it's not really an issue that needs to be fixed.

The real issue here is that some browsers does not handle that type of codec switching well. The RxPlayer thus might need to first "reload" before doing so (why it doesn't work/doesn't seem to reload here is another issue to look at as you noted).

From here, we're not sure how to move forward, so that's what we need to discuss in the RxPlayer team. For example, a solution might be to allow an application to tell the RxPlayer that is should keep those as several "tracks"/Adaptations so they cannot be switched between one another without an explicit setVideoTrack call.

Thanks for your time. For now the patch I pushed totally fit my needs. I know SmartTVs are a bit exotic ;-)

Yes, they sometimes showed surprising behaviors! Though we tried to handle them better recently (from the end of last year).

Tizen is one of the TV platform we aim to fully support (the RxPlayer runs in production on them through the Canal+ application).

peaBerberian avatar Jun 18 '21 16:06 peaBerberian

After investigation, the missing reloading issue is also because of mixing codecs in adaptation.

in get_adaptation_switch_strategy.ts

function hasCompatibleCodec(
  adaptation : Adaptation,
  segmentBufferCodec : string
) : boolean {
  return adaptation.getPlayableRepresentations().**some**(rep =>
    areCodecsCompatible(rep.getMimeTypeString(), segmentBufferCodec));
}

In my case the will be some representations with compatible codecs. So I still I have to clarify with the one in charge of streams. Right now, is more than busy ! So that's not priority.

el-gringo avatar Jun 19 '21 14:06 el-gringo