[Feature Request] ScreenCast: add option to add a custom button

[grulja]

Currently in Chrome when sharing a screen there are two dialogs involved. One is Chrome dialog with additional option to share a tab and our portal dialog with options to share a screen/window. We would like to get rid of the Chrome dialog and work only with the portal one, but problem is that users won't have option to share a tab anymore.

One suggested solution was to add an additional button into the portal dialog. E.g. [Share tab]. This would act as [Cancel] button, but besides just closing the portal dialog, it would send a different response (currently we have 0 = Success, 1 = User cancelled, 2 = Cancelled for other reason) so Chrome will know that their specific button was picked and can offer the user to share a tab instead.

I would like to know your opinions or alternative solutions (if you have any).

cc: @jadahl @allanday

[GeorgesStavracas]

I don't think we should add any kind of app-specific flags to the portal. This seems like Chromium doesn't want to change its assumptions about what it can do with the environment, and this flag is an attempt to turn the screensharing dialog into Chromium's preview dialog. This won't work. Chromium & family cannot preview windows and monitors on Wayland.

[grulja]

I don't think we should add any kind of app-specific flags to the portal. This seems like Chromium doesn't want to change its assumptions about what it can do with the environment, and this flag is an attempt to turn the screensharing dialog into Chromium's preview dialog. This won't work. Chromium & family cannot preview windows and monitors on Wayland.

I'm not sure you got it right what this is about. This is not about showing previews or turning the portal dialog into Chrome's dialog. This is about adding a simple button that on response would show their dialog so they can do their own stuff in case user don't want anything from our portal.

[grulja]

I understand you don't want to add app specific content into our portal dialogs, but I would oppose that some portals already allow something like this, for example choices option in the FileChooser portal allowing clients to add comboboxes.

I'm just trying to figure out how our dialog can co-exist with the Chrome's one or how we can substitute it so the user is not presented with two dialogs where functionality of one overlaps the other.

The button would be useful in following scenarios:

1. Chrome's dialog is not used and user is presented only with our dialog. In this case Chrome would like to also give the user an option to share a tab, but this cannot be integrated into our portal, therefore we would add a button e.g. [Share tab] which would close our dialog and Chrome would show their dialog for tab sharing instead.
2. Chrome's dialog is opened first giving option to either [Share screen] or [Share tab], but even in case the user mistakenly picks [Share screen] he would want get back somehow and clicking on [Cancel] in our dialog is not giving a clear answer whether this will bring back the option to share a tab or cancells whole share request.

[allanday]

The ideal solution for users is to show all the things they might want to share (tabs, windows, screens) in the same UI. Portals prevent us from doing this but, nevertheless, it is still necessary to allow users to go back and forth between the different things that they might want to share. Without this, it will be a frustrating experience.

I think we have two possible interaction flows, therefore:

Portal dialog first

1. User presses "share screen" on a web page
2. Portal dialog appears with: i. screen previews, ii. window previews, iii. "share a tab" button (opens the chrome dialog), iv. cancel button
3. User presses "share a tab"
4. Chrome dialog appears with: i. tab previews, ii. "share a screen or window" button (returns to the portal dialog), iii. cancel button

Chrome dialog first

1. User presses "share screen" on a web page
2. Chrome dialog appears with: i. tab previews, ii. "share a screen or window" button (opens the portal dialog), iii. cancel button
3. User presses "share a screen or window"
4. Portal dialog appears with: i. screen previews, ii. window previews, iii. "share a tab" button (goes back to chrome dialog), iv. cancel button

Both approaches are essentially the same in terms of UI. The portal dialog needs a button to open the Chrome dialog, or the Chrome dialog needs a button to open the portal dialog. Without that, you end up with a situation where the user has to cancel the whole process and start from scratch, if they decide they want to share something that was presented at the previous step.

[swick]

In theory we could let the application create pipewire streams for tabs, or anything else really, and then show those in the portal UI. This would even have the advantage that other apps could then consume those streams as well.

[allanday]

In theory we could let the application create pipewire streams for tabs, or anything else really, and then show those in the portal UI. This would even have the advantage that other apps could then consume those streams as well.

That would be amazing!

[grulja]

In theory we could let the application create pipewire streams for tabs, or anything else really, and then show those in the portal UI. This would even have the advantage that other apps could then consume those streams as well.

I don't really think this is a doable solution. First of all, the list of tabs is presented as a list and only once you pick one, you get a preview. Also, can you imagine creating tons of PipeWire streams and presenting them in our dialog? This would not be a great experience. You basically want to do portals the other way around, providing us access to what Chrome can share from inside. Btw. how is this different from showing just a list of tabs which has been considered as a no-go.

[swick]

First of all, the list of tabs is presented as a list and only once you pick one, you get a preview

Is that in firefox?

Also, can you imagine creating tons of PipeWire streams and presenting them in our dialog? This would not be a great experience.

Yeah, I didn't mean the app should just create actual pipewire streams whenever it has something that it could share, but announce to the ScreenCast portal that it has something it could share, with all the metadata we need to show it in the portal UI.

Btw. how is this different from showing just a list of tabs which has been considered as a no-go.

Giving the portal the metadata about possible streams it can share is different than allowing any app to create custom UI in the portal or having a case specifically for sharing tabs.

[grulja]

First of all, the list of tabs is presented as a list and only once you pick one, you get a preview

Is that in firefox?

Chrome/Chromium:

Btw. how is this different from showing just a list of tabs which has been considered as a no-go.

Giving the portal the metadata about possible streams it can share is different than allowing any app to create custom UI in the portal or having a case specifically for sharing tabs.

That's not what I would like to. I would be in favor adding a new tab into the portal dialog that would show list of tabs from Chrome. Like having options in the portal dialog to either share [Screens] [Windows] [Tabs] and each would be presented by a list. Screens and windows obviously can be presented as previews, but in case of tabs we would just show a list. But I believe this solution was not accepted very well as it adds app specific information into our dialog.

Also, I don't think it's doable from the technical perspective. All the portal and PipeWire implementation is in WebRTC, which interacts with Chrome through their public API. Sure, the API in WebRTC can be extended, but WebRTC don't have access to Chrome's tabs and I believe it would be really hard to make them work together. We would be also adding API to WebRTC that would be Chrome specific and other browsers (e.g. Firefox) would need a different one so I believe from this view it's a no-go.

[swick]

I have no clue about WebRTC and Chrome internals so bear with me.

The idea would be that there is a ScreenCastProvider portal which Chrome could use to tell the ScreenCast portal about the tabs that could be recorded. This information could be something like

{
  name: "Tabs",
  sources: [
    {
      id: "tab3156",
      name: "YouTube Music",
      icon: fd,
    }
  ],
}

And when the user opens a ScreenCast portal this would be shown somewhat like in your picture. If a tab is selected the ScreenCast portal asks the ScreenCastProvider portal for the corresponding pipewire stream so it can preview it. The ScreenCastProvider in turn asks the app for that stream.

The Chrome would then use the ScreenCastProvider portal, I guess, and WebRTC gets a pipewire stream from the ScreenCast portal no matter what the user selects.

[jadahl]

Just want to point out that on demand app provided PipeWire streams for previews for things the share can internally share is a much much much larger task than tweaking the cancel button. I'm not sure it's something that important to spend a large effort on.

If the cancel button was to be tweakable, it would be good to avoid allowing it to say "Kill a kitten" or something, e.g. a set of predefined options.

[allanday]

I've done some exploratory mockups, to see what the screen sharing dialog could look like with:

• an application-provided hint text
• thumbnails for the windows and screens
• an optional "share tab" button

From a UX perspective, each of these three changes can be considered independently. I think that they all work. The thumbnails in particular are a major improvement over what we have currently.