feat: get audio through `getDisplayMedia` or `tabCapture`

[WofWca]

• https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getDisplayMedia
• https://github.com/vantezzen/skip-silence/issues/123
• https://github.com/WebAudio/web-audio-api/issues/2453#issuecomment-940058417

I believe the only benefit would be the cross-origin circumvention (#47). But since it's circumvention, it could be bad for cross-origin security, so not sure if we need this. At least this could be made as an option.

[vantezzen]

Just FYI some browser-quirks for this if you want to implement it:

• tabCapture is Chrome-only and doesn't exist in Firefox
• Chrome mutes the tab completely when using preferCurrentTab: true with getDisplayMedia (https://bugs.chromium.org/p/chromium/issues/detail?id=1317964&q=preferCurrentTab&can=2). Due to this, users need to manually select the tab from the full list. This should be fixed soon though
• I tried requesting the deviceMedia from the background script but Chrome seems to randomly stop the capture for some reason - due to this, only running in the content script seems to be stable
• Firefox doesn't implement audio streams for displayMedia at all (https://bugzilla.mozilla.org/show_bug.cgi?id=1541425), so neither method work for Firefox
  • "I think this is low priority for us at the moment. This is a MAY in the spec"

it could be bad for cross-origin security

Could you explain how it is bad for cross-origin security? From my understanding, especially with tabCapture the complete capturing would be separated from the page which could actually improve security due to context splitting.

[WofWca]

Thanks for the insights!

Could you explain how it is bad for cross-origin security?

If a malicious site has a media element with a cross-origin source, in theory it should not be allowed to learn its contents, but some of it (volume levels) can be inferred based on observations of playbackRate changes caused by the extension. But, as I said in #47, some, more crucial, information can already be leaked, even without any extensions (see this), but, on a global scale, I think cross-origin access should not be possible in the first place and I don't think implementing such approach is worth making security a little worse. Especially if we were to hope that the cross-origin access (without CORS headers) is gonna be removed (which would solve the issue for our extensions).

[WofWca]

is worth making security a little worse

I understand that it's a very minute thing and it's very unlikely that someone's actually gonna try to exploit this in the next year or so, but still.

[vantezzen]

Ah, I see, thank you for the explanation! You're right, since the page can still access the playback rate that does leak some information.

But yeah, it would need to be a very specific attack scenario for this to be more useful than current browser-level data leaks, as the extension needs to be installed and activated. But I'll definitely keep that in mind!