StreamSwitcher
StreamSwitcher copied to clipboard
meething
What browsers are supported?
Right now I'm trying to support every browser except Safari. It looks like Firefox needs mozCaptureStream or else this breaks. If I change the code to use that then it breaks const stream = blackSilence(); trying to get the stream with a TypeError.
I'll keep seeing if I can get it to work but do you know of any other issues with Firefox or other browsers?
No I am not aware of any other issues on other browsers. We probably need some logic to support other browser along with chrome.
We can check the browsee upfront and than decide what we need to use.
blackSilence()
Can be adjustes/extendes to support mozilla browser
I can take a look to see if I can figure it out.
Do you know if this method with the caprureStream has any degradation in audio or video quality?
No quality etc is handled by your constraints so I dont expect lower quality.
Have you seen this before? https://github.com/muaz-khan/MultiStreamsMixer
Not sure how the implementation is different.
I know it and its widely used. But we solve different issues.
The multistreammixer uses canvas to layout multiple streams into on stream by placing them on a canvas. So you can have a sharescreen with a small webcam on the bottom right corner for instance.
What StreamSwitcher solves is switching full streams without need to pause video and keep recording.
If you are looking for something to mix two streams into one have a look at
https://github.com/bbc/VideoContext
That is wat more performant as it uses webgl I believe. So if you do heavy lifting and want to mix streams you best go is to use bbc VideoContext.
See her for my repo demo to add lots of video into one stream using videocontext
https://qvdev.github.io/video_mux_rec/all.html
And here is the repo
https://github.com/QVDev/video_mux_rec
What wont work with the one you mentioned due to canvas
@trackedsupport also see https://qvdev.github.io/video_mux_rec/
that is screen share with smaller webcam stream at right bottom corner
And that you can record as well and has good performance
oh dang, I thought the stream mixer would allow me to switch streams on the mixer while keeping the same stream with the recorder. All this stuff is new to me.
I want the ability to switch the video stream to a different camera or to a screen share while still recording locally in high quality. Right now with Iris there's an issue that if you switch your camera or mic during the recording it stops and I want to be able to add screen share soon too.
@trackedsupport than StreamSwitcher is what you want. It avoids the mediarecorder to stop
Alrright thanks for the help. Ill focus on getting this to work on other browsers.
First thing I noticed was that the supported method checks HTMLVideoElement.prototype.captureStream but the captureStream used in the code is on a canvas element so it should be HTMLCanvasElement.prototype.captureStream right?
StreamSwitcher does not use canvas so can directly do capture stream on the videobject as in the demo
oh okay - I guess I just don't understand enough yet! I only said that because I saw this.
https://github.com/meething/StreamSwitcher/blob/5e5ab141eff1d84f65e362bad45817db9e3d68df/js/ReplaceableMediaStream.js#L31
Right now im not directly supporting Safari but I would like to since its so close to working.
If I get this to work with Firefox, which I believe it can, it still wont work with Safari right?
https://caniuse.com/?search=captureStream
oh okay - I guess I just don't understand enough yet! I only said that because I saw this.
https://github.com/meething/StreamSwitcher/blob/5e5ab141eff1d84f65e362bad45817db9e3d68df/js/ReplaceableMediaStream.js#L31
That canvas is the blackstream empty one not from the videos. The empty / silence stream is for initial.setup
Right now im not directly supporting Safari but I would like to since its so close to working.
If I get this to work with Firefox, which I believe it can, it still wont work with Safari right?
https://caniuse.com/?search=captureStream
Indeed capturestream is not supported by safari. Question if it ever will be
oh okay - I guess I just don't understand enough yet! I only said that because I saw this. https://github.com/meething/StreamSwitcher/blob/5e5ab141eff1d84f65e362bad45817db9e3d68df/js/ReplaceableMediaStream.js#L31
That canvas is the blackstream empty one not from the videos. The empty / silence stream is for initial.setup
So then it has to support HTMLCanvasElement.prototype.captureStream as well right?
Yes or you can change that code to a short black video or even a black gif I think that it wont need the canvas
But you need capture stream for the video object as well so eliminating this canvas capturestream wont help on the long run
Yea I see your point.
Just to let you know I did play around with the MultiStreamMixer and it did work. The height and width require some setting and I'm not sure if the quality is as good. Also when you reset the video there's no longer any audio even if I add the stream.
Is there a downside in replicating this way of updating the streams? It's like what you said, it's built for another purpose but seems like it could be good with it working on all browsers.
So I had a working solution with the MultiStreamMixer through RecordRTC but that thing eats up so much memory. I have an older macbook pro and it crashes it after a few minutes eating gigs of memory on Chrome.
Going to trash it since it won't work. I guess I'll circle back around to this and see what I can learn and improve.
I'm trying to circle back to this. Altho im not sure what I am doing wrong now.
var local_media_stream = new MediaStream();
await navigator.mediaDevices.getUserMedia({ audio: this.audio_options, video: this.exact_video_options })
.then(function(camera) {
local_media_stream.replaceVideoTrack(camera.getVideoTracks()[0]);
local_media_stream.replaceAudioTrack(camera.getAudioTracks()[0]);
})
local_media_stream.getTracks() #returns []
but if I inspect the local_media_stream with local_media_stream.videoSender.track that is correct. The problem is that im trying to record with the MediaRecorder.
recorder = new MediaRecorder(local_media_stream)
No that didn't work. I think before I used a hidden video element on the page to copy your example. But when I record local_media_stream.remoteStream then an empty file is created.
I added this to get it to work on Firefox. I also got this to work when I used a video html object and just hid it from the webpage. Im going to test for Safari to run our old way and use this for any browsers that support capture stream. Thanks for your work!
if (HTMLVideoElement.prototype.mozCaptureStream){ HTMLVideoElement.prototype.captureStream = HTMLVideoElement.prototype.mozCaptureStream; }
Glad you found the issue feel free to open a pull request so other developers can benefit from it as well. Great work 👍
Im still messing around. I got it working on Safari. It appears you can save local_media_stream.remoteStream with the media recorder as long as you do local_video.srcObject = local_media_stream.remoteStream; first. Not sure why but I'm still looking. If you do it this way, you don't need the HTML element capture stream which safari doesn't have.
I did have to add this also btw to get it to work with Safari.
var AudioContext = window.AudioContext;
if (typeof AudioContext === 'undefined') {
if (typeof webkitAudioContext !== 'undefined') { AudioContext = webkitAudioContext; }
if (typeof mozAudioContext !== 'undefined') { AudioContext = mozAudioContext; }
}
Am I correct that replaceVideoTrack does not use the fps, width, or height from the stream? I have a 1280x720 24fps stream but for some reason its 320x180 30fps.
Edit: It appears to slowly grow. First 320, then 540, then 720. Weird?
example
: