Get appended audio bytes in an event

[danrossi]

There doesn't seem to be a method or documentation to get the remuxed appended audio bytes. hls.js has an event when bytes are appended providing the bytes in the event. But it doesn't seem that is the case for vhs possibly requiring to use hls.js as a tech somehow.

The only possible way is to patch this method ?

https://github.com/videojs/http-streaming/blob/bd810ea88c8604ebbdd3d886a75984dfee9b5b4b/src/source-updater.js#L585

[danrossi]

It seems to be an oversight. There is an event called before appending with not really useful information on the segment without the merged bytes with the init segment bytes. It requires to be patched like this sadly. MediaSource has no api to obtain the bytes from the sourcebuffers so an oversight also or a security issue no idea.

It's required for a HLS audio api hack for Safari.

 player.ready(async () => {

        const vhs = player.tech().vhs;

        if (vhs) {
            const playlistController = player.tech().vhs.playlistController_,
            oldAppendBuffer = playlistController.sourceUpdater_.appendBuffer.bind(playlistController.sourceUpdater_);

            playlistController.sourceUpdater_.appendBuffer = (data) => {

                oldAppendBuffer(data);

                if (data.type == "audio") {
                    console.log("data ", data);
                }
            };
        }
        

        
    });

[danrossi]

It works for a while but then the init segment isn't in the data being appended. So needs an event to get the seperated init segment.

[danrossi]

This patching is required to get the initsegment to merge back in

 const playlistController = player.tech().vhs.playlistController_,
                        //oldAppendBuffer = playlistController.sourceUpdater_.appendBuffer.bind(playlistController.sourceUpdater_),
                        oldAppendData = playlistController.mainSegmentLoader_.appendData_.bind(playlistController.mainSegmentLoader_);


                    playlistController.mainSegmentLoader_.appendData_ = (segmentInfo, result) => {
                        oldAppendData(segmentInfo, result);

                        

                        if (result.type == "audio") {

                            //console.log("result ", segmentInfo, result);

                            let newBuffer = mergeBuffers(result.initSegment.buffer, result.data.buffer);

                            appendBuffer(newBuffer);
                        }
                    };