Detect when minimizing a window pauses capture (was: Capability polling - capture-while-minimized)

[eladalon1983]

Recall what the spec says about minimized surfaces:

A display surface that is being shared may temporarily or permanently become inaccessible to the application because of actions taken by the operating system or user agent. What makes a display surface considered inaccesible is outside the scope of this specification, but examples MAY include a monitor disconnecting, window or browser closing or becoming minimized, or due to an incoming call on a phone.

So minimizing a window might pause the capture.

Suppose a Web developer wants to warn users if minimizing windows would stop the capture; we have an example of such a developer here:

We need to give the user a hint that minimizing the window will cause the screen to not capture data

Currently, there's no way for this developer to know if the warning should be issued. It would be nice if developers could query that. Ideally before getDisplayMedia() is ever called, but possibly also after, via a MediaStreamTrackCapability.

[youennf]

The current wording is unclear, is there a chance we can standardize more of the behavior in this area? How about the following rules:

• UA ends the track when the window gets closed
• UA mutes/unmutes the track when the window gets minimized/unminimized

Currently, there's no way for this developer to know if the warning should be issued

If we standardize more of the behavior, the developer will know which warnings to provide without querying a new capability or without doing UA string checks.

[eladalon1983]

UA ends the track when the window gets closed

Makes sense to me.

UA mutes/unmutes the track when the window gets minimized/unminimized

I am not sure that's desirable. Especially on Windows, I could see a user tapping the shared window's taskbar entry to just minimize a window and get it out of the way. It would not indicate a user's desire to stop the capture.

If we standardize more of the behavior, the developer will know which warnings to provide without querying a new capability or without doing UA string checks.

The OS-level APIs for capture might differ in whether they allow a minimized window to be captured.

[jan-ivar]

UA ends the track when the window gets closed

This makes sense to me as well.

UA mutes/unmutes the track when the window gets minimized/unminimized

I am not sure that's desirable. Especially on Windows, I could see a user tapping the shared window's taskbar entry to just minimize a window and get it out of the way. It would not indicate a user's desire to stop the capture.

I agree. But it sounds like we agree that if minimizing windows would stop the capture, then the UA MUST fire mute/unmute while minimized.

Wouldn't this solve the OP use case?

[jan-ivar]

Hmm the existing wording in https://w3c.github.io/mediacapture-screen-share/#hidden-display-surfaces seems to cover this fairly well. @youennf can you elaborate on what is not clear there?

[youennf]

@youennf can you elaborate on what is not clear there?

I was probably looking only at the paragraphe above. The paragraphs in the spec after the quoted one above are indeed mentioning when to fire mute/unmute/ended events.

[eladalon1983]

I agree. But it sounds like we agree that if minimizing windows would stop the capture, then the UA MUST fire mute/unmute while minimized.

Wouldn't this solve the OP use case?

It would almost solve the problem. Missing functionality includes:

1. No way for the app to know the the track would be muted if the window minimized, until the window is in fact minimized. Possibly we could defer fixing this particular issue.
2. No way for the app to know why the track was muted. Window minimized? It's just one of several possibilities. But if we had MuteReasons...