element-hq
Screensharing should share audio on selected window
Is your suggestion related to a problem? Please describe. When using screen sharing, the sound for the selected window is unavailable.
Describe the solution you'd like The audio from the selected window should be shared. If possible, this should also be available from sharing the entire screen, however I am not sure if there is an API available for taking all sound output.
I don't see any API in browsers for this from a quick glance, it feels like internal browser behaviour unfortunately, do you know of any web apps which have this feature?
@t3chguy, not sure about browsers, but Electron can definitely do this
I am wondering if this should be merged into #12652. When that gets done properly (multiple WebRTC streams), this shouldn't be a problem to implement
The electron docs suggest that audio is an all-or-nothing deal, so wouldn't be able to get it from a certain application
To capture video from a source provided by desktopCapturer the constraints passed to navigator.mediaDevices.getUserMedia must include chromeMediaSource: 'desktop', and audio: false.
To capture both audio and video from the entire desktop the constraints passed to navigator.mediaDevices.getUserMedia must include chromeMediaSource: 'desktop', for both audio and video, but should not include a chromeMediaSourceId constraint.
For this, we are going to need to give the user the ability to change the volume of individual feeds, something similar to what Discord has. @gaelledel, could you please take a look into this. I think we could just go with a feed right-click context menu which would have a Mute button and a volume slider. Does that sound good to you? Hopefully, designing this should be quite straightforward
Just saying, this is one of the major issues with matrix for me! I'd love to see this implemented so I can call friends on matrix more! I love that there's screen sharing now, but this would make it perfect. 💜
I would like to say that Discord in the Browser Version on Chromium based Browsers is able to transfer audio for screen casts, if you select a specific window. (Element Desktop is Electron based just like Discord Desktop)
I would like to say that Discord in the Browser Version on Chromium based Browsers is able to transfer audio for screen casts, if you select a specific window. (Element Desktop is Electron based just like Discord Desktop)
afaik Discord uses an undocumented hidden Windows API for this behaviour directly. And just recently got support for it on MacOS via I assume their platform-specific API's as well.
I would like to say that Discord in the Browser Version on Chromium based Browsers is able to transfer audio for screen casts, if you select a specific window. (Element Desktop is Electron based just like Discord Desktop)
afaik Discord uses an undocumented hidden Windows API for this behaviour directly. And just recently got support for it on MacOS via I assume their platform-specific API's as well.
I don't quite know or found the code for this feature but I found the following in the electron documentation. There is something mentioned about audio recording: https://www.electronjs.org/docs/latest/api/desktop-capturer This electron app is also using audio: https://github.com/wulkano/Kap/search?q=audio
Maybe you could get something out of it. 🙂
Capturing systemwide audio, selected window audio and/or microphone audio is possible, I have found the documentation here on MDN. It depends on what the user agent allows but from what I understand there are only a few issues with this, like the ones documented by Electron.
We are aware that this is technically possible though the issue is awaiting design work which is necessary for it to progress
In chromium-based browsers, you can share a tab. There the audio is automatically provided by the browser, but therefore the audio has to be enabled as described at the MDN Link https://github.com/vector-im/element-desktop/issues/791. It would be a nice starting point for audio transmitting in screencasts.
This will be in the next version of Element Call
Amazing, but what is Element Call exactly, is it the desktop/mobile client or something separate?
This will be in the next version of Element Call
Amazing, but what is Element Call exactly, is it the desktop/mobile client or something separate?
It's this: https://element.io/blog/introducing-native-matrix-voip-with-element-call/
This will be in the next version of Element Call
Amazing, but what is Element Call exactly, is it the desktop/mobile client or something separate?
It's this: https://element.io/blog/introducing-native-matrix-voip-with-element-call/
I’ve been using the testing environment of Element call and as I understand it, it is what runs natively in the 1:1 chats of the current Desktop version but there are still a lot of bugs when trying to share audio during screen share. Is there any recent update to this?
@kazin-essen Element Call doesn't run natively in 1:1 calls. That stack predates Element Call.
@kazin-essen Element Call doesn't run natively in 1:1 calls. That stack predates Element Call.
Ohh, I see, do 1:1 calls still use Jitsi then, or some version of Element Call? To be honest I’ve never been quite sure what the difference was between call.element.io being tested, Video Call Beta in Labs and the 1:1 E2EE chat. Group Spaces chat uses Jitsi still right? The goal is to have which version be the final one? Call? Current 1:1? Something new?
1:1 calls use the current specced version of VoIP in Matrix, Element Call uses a more experimental one. Element Call is working to replace Jitsi in Element products.
This will be in the next version of Element Call
Is it live yet?
Yep :)
Apologies if I failed to understand anything but will this be implemented in element client as well?
I've just recently started to use Element and miss this feature. Do you have a plans to implement it? Have to use Skype so far :( Thnx
When this is done there should be no real reason to switch to a different application. I suggest looking at the audio screen sharing of third party Discord applications such as Vencord or Webcord, who used their own solutions to upend Discord's sloth towards the issue on GNU/Linux-based systems. I believe the repo, for Vencord's implementation, is called Venmic. If you know about this already, I apologize for the redundancy. This is not a "hurry up" message. Any perceived urgency is not as such. I'm just saying there's framework of ideas to work with out there if you guys are stumped right now, especially since they did such a heavy duty modification in a closed source application seeing the same issues on Linux, also using Electron. As someone learning RE, I find that significant in of itself, a treasure trove even, for open source projects who want to devise a plan for this themselves. Idk. good luck 🌈
When this is done there should be no real reason to switch to a different application. I suggest looking at the audio screen sharing of third party Discord applications such as Vencord or Webcord, who used their own solutions to upend Discord's sloth towards the issue on GNU/Linux-based systems. I believe the repo, for Vencord's implementation, is called Venmic. If you know about this already, I apologize for the redundancy. This is not a "hurry up" message. Any perceived urgency is not as such. I'm just saying there's framework of ideas to work with out there if you guys are stumped right now, especially since they did such a heavy duty modification in a closed source application seeing the same issues on Linux, also using Electron. As someone learning RE, I find that significant in of itself, a treasure trove even, for open source projects who want to devise a plan for this themselves. Idk. good luck 🌈
Yeah, pretty much all the work you need to implement screensharing is in these two repos: https://github.com/maltejur/discord-screenaudio https://github.com/edisionnano/Screenshare-with-audio-on-Discord-with-Linux
Which is what Vencord used to make their solution work https://github.com/Vencord/venmic
