node-ebml icon indicating copy to clipboard operation
node-ebml copied to clipboard

How would I use node-ebml with MediaRecorder to stream audio from a browser mic to server?

Open rmtuckerphx opened this issue 1 year ago • 2 comments

I'm using the following code to get an audio stream from the web browser's microphone:

let chunks = [];
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
mediaRecorder = new MediaRecorder(stream);
mediaRecorder.ondataavailable = (e) => {
    chunks.push(e.data)
    // wrap e.data in MKV container and send to server KVS with POST /putMedia
}
mediaRecorder.start(1000); // call ondataavailable (send chunk) every second

In order to stream the data to the server with a Kinesis Video Streams stream (can be audio only or other time series data), it needs to be in a Matroska (MKV) container:

https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_PutMedia.html

I know the REST call that needs to be made to send the data to KVS, I'm just not sure about the packaging with MKV.

How would I use the node-ebml library on my web page to encode each data chunk in an MKV container?

rmtuckerphx avatar Feb 12 '23 15:02 rmtuckerphx

WebM is Matroska, just with the additional constraint of using fewer codecs. So, if you use a WebM content type when setting up your Media Recorder, you're good to go.

bradisbell avatar Feb 12 '23 17:02 bradisbell

I'm using the following code to get an audio stream from the web browser's microphone:

let chunks = [];
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
mediaRecorder = new MediaRecorder(stream);
mediaRecorder.ondataavailable = (e) => {
    chunks.push(e.data)
    // wrap e.data in MKV container and send to server KVS with POST /putMedia
}
mediaRecorder.start(1000); // call ondataavailable (send chunk) every second

In order to stream the data to the server with a Kinesis Video Streams stream (can be audio only or other time series data), it needs to be in a Matroska (MKV) container:

https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/API_dataplane_PutMedia.html

I know the REST call that needs to be made to send the data to KVS, I'm just not sure about the packaging with MKV.

How would I use the node-ebml library on my web page to encode each data chunk in an MKV container?

Yes, that's what Loom does. I've been researching this too. I don't know how Loom implements it. I asked questions on stackoverflow, but I didn't seem to get a good answer! If you have a better solution, please contact me, thank you very much!

gankaicode avatar Mar 06 '23 10:03 gankaicode