getAudioContext + stream + MediaRecorder generate undefined Blobs

[doriclaudino]

Hello!

I'm trying to combine the audio+canvas in a single MediaRecorder. I did using canvas + audio elements (and caputureStream API), but it's not my goal.

Following some tutorials and examples from mozilla, it can be achieve doing audioContext.createMediaStreamDestination and then get the stream prop.

The getAudioContext() + MediaRecorder for some reason I got undefined Blobs

snippets:

function exportAudio2() {
  stream = new MediaStream();
  let ctx = getAudioContext();
  let dest = ctx.createMediaStreamDestination();

  let audioStream = dest.stream;
  var audioTracks = audioStream.getAudioTracks();
  var firstAudioTrack = audioTracks[0];
  stream.addTrack(firstAudioTrack);

  recorder = new MediaRecorder(audioStream, (options = {}));
  recorder.onerror = (evt)=>console.log(evt)

  //no data availble
  recorder.ondataavailable = (evt)=>console.log(evt)

  recorder.start();
}

output from ondataavailable:

bubbles: false
cancelBubble: false
cancelable: false
composed: false
currentTarget: MediaRecorder {stream: MediaStream, mimeType: "", state: "inactive", onstart: null, onstop: ƒ, …}
data: Blob
size: 0  //here
type: ""
__proto__: Blob
defaultPrevented: false
eventPhase: 0
isTrusted: true
path: []
returnValue: true
srcElement: MediaRecorder {stream: MediaStream, mimeType: "", state: "inactive", onstart: null, onstop: ƒ, …}
target: MediaRecorder {stream: MediaStream, mimeType: "", state: "inactive", onstart: null, onstop: ƒ, …}
timeStamp: 8138.389999978244
timecode: 1587931417884.0781
type: "dataavailable" //here
__proto__: BlobEvent

Alternative solution: Chrome-Audio-Capturer

Seems is possible to capture the whole page audioContext as stream using chrome.tabCapture.capture

chrome.tabCapture.capture({audio: true}, (stream)
const liveStream = stream;
const audioCtx = new AudioContext();
const source = audioCtx.createMediaStreamSource(stream);
let mediaRecorder = new Recorder(source);

Anyone thoughts?

[andreasvirkus]

@doriclaudino did you ever figure this out? I have a similar situation, where I would like to get a stream of a p5.Gain() and add it to the video stream of the canvas. This is what I've tried so far:

const options = { mimeType: 'video/webm;codecs=opus', videoBitsPerSecond: 10000000 }
const videoStream = document.querySelector('canvas').captureStream(60)
const destination = masterGain.ac.createMediaStreamDestination()
const audioStream = destination.stream

// approach A
const stream = new MediaStream([videoStream, audioStream])
const recorder = new MediaRecorder(stream, options)

// approach B
videoStream.addTrack(audioStream.getAudioTracks()[0])
const recorder = new MediaRecorder(videoStream, options)

But neither of these result in the audio getting streamed into the recording.