electron

electron copied to clipboard

I am using the desktop sharing feature of the Electron desktopCapturer on our demos. Some of our demos require to share audio of our application. When we share the desktop with audio on desktopCapturer electron client participants start to hear their own voice.

const constraints = {
  audio: {
    mandatory: {
      chromeMediaSource: 'desktop'
    }
  },
  video: {
    mandatory: {
      chromeMediaSource: 'desktop'
    }
  }
}

Current behavior:

When screen share enabled with audio participant sounds echoing back to themselves.

Expected Behavior:

Only system sounds should be broadcasted to the conference.

Expected Solution:

Behaviour should be converted as same as chrome web client.

Steps:

1- Using desktopCapturer Electron 2- Share screen with audio 3- Another participant starts to talk. (As a result, hears themself)

Environment:

Windows 10 Electron ^9.2.1 desktopCapturer Electron

Same issue here. Also trying to capture the audio for one application/window results in getting the entire system audio instead of the audio for that specific application.

Thanks for reporting this and helping to make Electron better!

Would it be possible for you to make a standalone testcase with only the code necessary to reproduce the issue? For example, Electron Fiddle is a great tool for making small test cases and makes it easy to publish your test case to a gist that Electron maintainers can use.

Stand-alone test cases make fixing issues go more smoothly: it ensure everyone's looking at the same issue, it removes all unnecessary variables from the equation, and it can also provide the basis for automated regression tests.

I'm adding the blocked/need-repro label for this reason. After you make a test case, please link to it in a followup comment.

Thanks in advance! Your help is appreciated.

Thanks for reporting this and helping to make Electron better!

Because of time constraints, triaging code with third-party dependencies is usually not feasible for a small team like Electron's.

Would it be possible for you to make a standalone testcase with only the code necessary to reproduce the issue? For example, Electron Fiddle is a great tool for making small test cases and makes it easy to publish your test case to a gist that Electron maintainers can use.

Stand-alone test cases make fixing issues go more smoothly: it ensure everyone's looking at the same issue, it removes all unnecessary variables from the equation, and it can also provide the basis for automated regression tests.

I'm adding the blocked/need-repro label for this reason. After you make a test case, please link to it in a followup comment.

Thanks in advance! Your help is appreciated.

I'm closing this as there's no minimal repro available. I'm happy to reopen if you're able to provide a more directed example that shows the issue.

@nornagon Hello there I created this repository https://github.com/Apak00/electron-ss-echo-problem starting from electron-quick-starter. As it requires a peerconnection to be established, there is a signaling server involved but it should have nothing to do with echo problem because all it does is really trading the sdp between peers. Also added reproduce steps, but if you have any problem with reproducing, I am more than happy to help.

I hope this helps you

Here’re some findings after struggling with this issue for months.

1. Using getDisplayMedia() in chrome doesn’t have echo issue, but it will have it too if echo cancellation was turned off.
2. Debug log shows getUserMedia() have no audio processing applied, but getDisplayMedia() does.
3. Audio processing was skipped since chrome want to create a direct-path and skipped it for screen capture device.
4. getUserMedia() was caught in above by its device type, getDisplayMedia() have different device type and escaped
5. Tried to made a mod to disable checking of screen catpure device type, it do works and echo was removed. But there're still lot of differences comparing to getDisplayMedia()

What I modified: third_party/blink/renderer/modules/mediastream/user_media_processor.cc:CreateAudioSource() Commented out blink::IsScreenCaptureMediaType(device.type)

Seems to work, should make a PR.

@codebytere

Team, any update on this issue.

@dx200010 If possible, can you please provide your code snippet or repo with working example.

@dx200010 If possible, can you please provide your code snippet or repo with working example.

Firstly, please make sure you have downloaded full source codes of electron by following official build instruction.

Find the UserMediaProcessor::CreateAudioSource() in third_party/blink/renderer/modules/mediastream/user_media_processor.cc and change below if-block

  if (blink::IsScreenCaptureMediaType(device.type) ||
      !blink::MediaStreamAudioProcessor::WouldModifyAudio(
          audio_processing_properties)) {

to

  if (//blink::IsScreenCaptureMediaType(device.type) ||
      !blink::MediaStreamAudioProcessor::WouldModifyAudio(
          audio_processing_properties)) {

Then build a modified electron from source, and use this modified electron to build your own application.

Now, you can enable echo cancellation by inserting the constraint like below.

navigator.mediaDevices.getUserMedia({
    audio : {
        mandatory: {
            echoCancellation: true,
            chromeMediaSource: 'desktop'
          }
    },
    video :
    {
        mandatory :
        {
            chromeMediaSource : 'desktop'
        }
    }
})	

@dx200010 Thanks a lot for the details, But unfortunately this did not work for me.

Could it be because of a certain electron version I need to checkout? the version I tried is : 12.0.1

@dx200010 Thanks a lot for the details, But unfortunately this did not work for me.

Could it be because of a certain electron version I need to checkout? the version I tried is : 12.0.1

Maybe... I have only tried this on 13.1.4, don't know if it's version specific or not.

@dx200010 Have you had a chance to try your custom electron build on this repo? https://github.com/Apak00/electron-ss-echo-problem

I point my package.json electron dependency to custom electron build like this devDependencies: { ... electron: "file:../electron/src/electron" }

Even though I am able to build the app with this, could that be wrong?

What is the right way of adding custom electron build to a project?

@Apak00 Maybe your app was still linked to an unmodified electron.

Try launching your app directly from your own built electron for testing, <Path to your electron source>\out\Release\electron.exe .

If it does works, then it's just packaging issue. Or, this workaround may just not always works.

Either way, would love to fix this upstream instead of having to build Electron which is quite time consuming for many.

I'm closing this as there's no minimal repro available. I'm happy to reopen if you're able to provide a more directed example that shows the issue.

hi，nornagon，Same issue here

Team, any update on this issue?

I can confirm that this workaround actually fixes the issue on my end. Right now we use a custom electron on our project with the workaround applied, but I would love to see an upstream solution.

Team, can we work on fixing this in the upstream solution?

Team, any further update will be much appreciated.

Any update on this issue?

Facing the same issue, is there any PR raised for fixing this?

I'd like to add that this is something that already works in some popular Electron apps (eg. Discord, Teams). It would be nice to have it implemented properly upstream as well.

Any update on this issue?

Team, any update will be much appreciated.

Have also encountered this issue. Am using Electron 15.0.0. Am going to attempt the above workaround involving creating a custom electron package. Wish there was even more steps/documentation on how to exactly do this.

+1 for requesting this to be fixed properly, in the official electron package, to avoid workarounds.

Team, any update will be much appreciated.

Any update on this issue?

Would be great to get this merged in

Any updates on this? Is there anything I could do to move this forward or help things along here?

Any update on this issue will be much appreciated.

Team, any update on this issue.

This issue is also affecting me. As the culprit seems to be in Chromium, I've opened an issue there: https://crbug.com/1286503

@mwcampbell fwiw your issue will get addressed 99999999x faster if you open a CL with your proposed change - i can also try to do it if you don't have bandwidth but i might not get to it for a bit

According to https://bugs.chromium.org/p/chromium/issues/detail?id=1286503#c8 this is an upstream issue and is a feature request, not a bug per se. I'm marking this blocked/upstream to reflect that.

After looking into this topic (and the associated issue opened for Chromium) and doing a lot of tests to find a possible workaround, for a few days now, there is a general question I'd like to ask. My impression is, that Chromium does nothing wrong. According to @dx200010, the function getDisplayMedia() works fine in the browser and also cancels the echo there. On Electron we have to use getUserMedia() with a specific constraint to achieve the same functionality (which lacks echo cancellation). I'm not doubting that there is a good technical reason for this, but when we consider this to be a Chromium issue, we do request from the Chromium team to implement a specific functionality (which they don't need for the browser itself) just to get it work in Electron. Thus I can understand that they do not consider this to be a bug and not high priority as well.

I'm not in the details of the Electron/Chromium code at all, but I'd just like to raise the question whether there is really no feasible way to take over the the getDisplayMedia() implementation.

BTW: I was able to extract the app code of MS Teams, because I wanted to know how they achieve audio sharing without echo. From what I could see, they created an own native module called "slimcore" (not to be confused with a module of the same name which was removed from npm registry because of malicious code!) which seems to handle this. If someone want's to have a look as well (and maybe find out more), here is how to enable the developer tools in Teams:

7x left-click on Teams tray icon. Then right-click and you get additional tray menu options which let you open the developer tools 😉.

If we definitely can't switch to getDisplayMedia(), wouldn't it be possible to add a temporary method to enable the workaround proposed by @dx200010 explicitly until a final solution is implemented? I could think of a:

• command line switch
• getUserMedia() constraint
• environment variable
• etc.

This would be very helpful!

I've just built Electron by myself with the proposed patch and I can confirm that it works like a charm with the demo app from @Apak00! Thanks for your great effort to discover this @Apak00 !

Guys, it would really help so much if you could implement this in the officially build. I'd suggest to simply evaluate the (in this case unused) echoCancellation: true constraint.

@codebytere, I could try to build a pull request, if you want me to, but I guess you guys will be 100 times faster than me. I have no experience in contributing to the Electron code base.

@Apak00, I guess you probably solved the issue with packaging your app with a custom Electron build meanwhile. I had the same problem. My impression is that electron-builder is not using the Electron binaries installed in node_modules folder. Instead it seems to just lookup the Electron version number in package.json, but then downloads that Electron version by itself from the server, thus making it impossible to use a private Electron build. I found a way to workaround this by using the win-unpacked folder, by replacing the Electron files by my self-built ones manually. Its not nice, but it works. So in case others ran into the same issue: https://stackoverflow.com/questions/72633870/how-to-use-self-built-electron-binaries-with-electron-builder If someone knows how to properly use a self-built Electron in a an app please let me know/post a better answer on Stackoverflow. ;-) Thanks!

I guess you are looking for 'electronDist' property on electron-builder configs, here; https://www.electron.build/configuration/configuration.html

I just ended up writing an entire separate media engine for my app in native C++. Unfortunately, Electron almost never fixes or adds anything unless some major company needs it.

@Apak00 @berkon I built my own electron with the suggested changes and ran my application from it. Although the stream.getAudioTracks()[0].getSettings() shows that echoCancellation: true, the participants of the stream still hear their own voice. Any thoughts on how to fix this? I am using Electron v13.6.9

@bzmaxat, did you see Apak00's post regarding the electronDist attribute? In my case that worked. If it doesn't, then I'd guess that something went wrong with the compilation. To be honest, I wouldn't trust the content of the echoCancellation attribute in this specific case.