Multiple controllers of remote playback

[pthatcherg]

Can we allow for multiple controllers of a remote playback session like we allow for presentations? If so, what would that protocol look like? What controls would be allowed?

[takumif]

We can allow agents to connect to an existing session without modifying Remote Playback API. It'd allow controlling playback from browser UI, for example. I think we'd want those agents to be able to stop the session, so it'd make sense to just allow full control.

Related: do we want a request message to terminate a session regardless of its type (Presentation, Remote Playback, or streaming)?

If we were to allow creating sessions that can refuse to be controlled by other senders, who should be able to specify this? The controlling page, the controlling agent, or the receiving agent?

---

If we want to let web pages connect to an existing session, we have a few options to do this:

• Allow RemotePlayback.prompt() to connect to existing sessions (by having the user select a receiver with an existing session in the browser UI)
• Create a new API that's similar to PresentationRequest.reconnect()

Questions:

• What pages do we allow reconnecting? Pages with the same origin as the original sender page? (Pages with the same origin as the media src probably wouldn't work, given the media may be served by a CDN)
• If the local and remote media have different attributes (e.g. src), do we override the local ones with the remote ones?

[pthatcherg]

We can allow agents to connect to an existing session without modifying Remote Playback API. It'd allow controlling playback from browser UI, for example. I think we'd want those agents to be able to stop the session, so it'd make sense to just allow full control.

Related: do we want a request message to terminate a session regardless of its type (Presentation, Remote Playback, or streaming)?

I doubt that's worth it. Having separate messages seems easy enough.

If we were to allow creating sessions that can refuse to be controlled by other senders, who should be able to specify this? The controlling page, the controlling agent, or the receiving agent?

Ultimately, the receiver chooses who it will allow to control it, so I'd say the receiver. But the initial controller could ask the receiver to do so, and the controlling agent may allow a controller page to ask it to ask the receiver to do so.

If we want to let web pages connect to an existing session, we have a few options to do this:

• Allow RemotePlayback.prompt() to connect to existing sessions (by having the user select a receiver with an existing session in the browser UI)

Would that be using a blank/empty HtmlMediaElement?

Seems like surprising/confusing overloaded behavior.

• Create a new API that's similar to PresentationRequest.reconnect()

On a blank/empty HtmlMediaElement? What if it's not empty? Does it change the source of the remote element?

Does this need an ID to refer to?

I'm in favor of this API over using prompt().

Questions:

• What pages do we allow reconnecting? Pages with the same origin as the original sender page? (Pages with the same origin as the media src probably wouldn't work, given the media may be served by a CDN)

What can Presentations do right now?

• If the local and remote media have different attributes (e.g. src), do we override the local ones with the remote ones?

I would say yes.

[takumif]

On a blank/empty HtmlMediaElement? What if it's not empty? Does it change the source of the remote element?

Maybe there can be two methods: one that always uses the controller's source, and another method that overwrites the local element with the receiver's media, if there already exists a session. This would let the site choose the behavior.

Alternatively, prompt() can handle all the cases, and the user can choose in the UA's UI which source to use. Then the controller page would know which by looking at the currentSrc (or src?) This would let the user choose the behavior.

We need to handle cases where we're using the receiver's source and the media types are different (<audio> vs <video>) between the controller and receiver elements. We can require the controller UA to prevent such reconnects, or throw an error. I think the former is a better UX.

What can Presentations do right now?

It requires that the reconnecting PresentationRequest already knows the receiver's URL: "Its presentation URL is equal to one of the presentation request URLs of presentationRequest". Its Remote Playback equivalent would be for the HTMLMediaElement to already have the media URL (or just the origin?) as a source, which I don't think is practical.

[pthatcherg]

On a blank/empty HtmlMediaElement? What if it's not empty? Does it change the source of the remote element?

Maybe there can be two methods: one that always uses the controller's source, and another method that overwrites the local element with the receiver's media, if there already exists a session. This would let the site choose the behavior.

That's an option. Another is one method with a good default (like pull is default and push is an opt-in override parameter).

The more I think about it, the more I think push is a better default and if you want pull, just clear your own source before you remote, or make a blank one. So I think I like "one method with push".

Alternatively, prompt() can handle all the cases, and the user can choose in the UA's UI which source to use. Then the controller page would know which by looking at the currentSrc (or src?) This would let the user choose the behavior.

That sounds too complex from a UX perspective.

We need to handle cases where we're using the receiver's source and the media types are different (

I think preventing reconnects make sense.

What can Presentations do right now?

It requires that the reconnecting PresentationRequest already knows the receiver's URL: "Its presentation URL is equal to one of the presentation request URLs of presentationRequest". Its Remote Playback equivalent would be for the HTMLMediaElement to already have the media URL (or just the origin?) as a source, which I don't think is practical.

Oh, that's interesting. In that case, we wouldn't have to deal with push vs. pull.

But why would this be impractical for remote playback but not for Presentation API?

[markafoltz]

I think this is an interesting topic. An agent could get a remote-playback-state-event to find out about remote playbacks that are currently active, and send a remote-playback-modify-request to control that playback. I suppose this could be used by a browser to control an existing remote playback, or a site to reconnect to an existing remote playback.

However, for the first case (browser control), I would lean towards defining a different protocol that is closer in alignment to the Media Session API as remote controls shouldn't depend on where the remote media comes from (local app, presentation, remote playback, mirroring, etc.) I will file a v2 issue for this.

For the second case (site reconnection), it will require some modifications to the Remote Playback API - I don't think we can swap out a site's media element with a remote playback without its involvement. I suggest continuing discussion in w3c/remote-playback#5.