[Feature request]: Exclusions enhancement - Pause while camera is in use

[e-t-l]

Version

• [X] I'm using version 1.7.0

Existing requests

• [X] I've checked Existing requests

What operating system are you using?

Windows

Operating System Version

Windows 10 v 20H2

Problem description

In addition to pausing Stretchly while a specific app is running, there are times users would want to pause Stretchly when accessing certain websites (e.g. browser-based video calling like Google Meet, Jitsi Meet, Zoom (browser), or Facebook Video Call).

Possible solution

While I don't believe there's any way for a JS app to detect the webpages open in a different browser*, there are programmatic ways to detect when the device's camera is in use. This StackOverflow question might be a good starting place.

Alternative solutions

*I could be wrong about detecting open tabs in other browsers. In Windows, for example, when a user hovers over an open browser's taskbar icon, it displays a preview with the name of the active tab. If there's a way to fetch this via Javascript, then we could implement per-website Stretchly exclusions, which would be even better.

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[hovancik]

I have tried the code from here: https://stackoverflow.com/a/42212720 in Chrome

It firsts asks for permission to use Camera, then starts using the camera and then says that camera us used.

Looks to me that that API only is about running camera in context of the current tab.

[e-t-l]

that API only is about running camera in context of the current tab.

Not sure what you mean.

My understanding was that (at least on Windows) only one program is allowed to access the camera at a time. If Stretchy has permission to use the camera but still cannot access it, then it would know that the camera is in use by another program. If it can access it, then it knows the camera is not in use. The code would just have to be tweaked so that it doesn't actually start using it.

[hovancik]

Well, what I say is that I have no idea and what I seem to see that API is not saying "wheter some process is using camera".

Just tried turning on camera on Teams, Chrome and Firefox at once and all of them are able to ask for camera and use it at the same time.

[samwzlim]

@hovancik It might be worth a shot taking a look at an app called 'Time Out' on the Mac App Store. When any video conferencing app is open, the breaks are automatically skipped. I'm sticking with Stretchly because I use both macOS and Windows, and Stretchly is cross-platform.

[hovancik]

I think that app is native so they have better options to use different APIs.

Just found, there's node package to detect Camera ON/OFF for MacOS but people say it's not really working: https://github.com/sindresorhus/node-is-camera-on/issues/3

[samwzlim]

I think pausing Stretchly while a video conferencing app is running would be more ideal as not all meetings require cameras to be turned on (and also easier to be implemented?). Not sure how that would be implemented but these are just my thoughts.

[hovancik]

@samwzlim that is implemented already: https://github.com/hovancik/stretchly#pauseresume-breaks-only-when-specific-command-is-running

[hovancik]

In ideal world, all those apps (Teams, Slack, Video player, Game, ..) should simply put DND/Focus mode ON when the meeting/game/whatever is on and Stretchly simply follows that. That's why there is OS-level dnd mode....

As I see it, all my code (and code of many other apps that try to detect bilions of things) is pointless excercise in big corporations being lazy and us doing the work for free...

[vvvlc]

I wanted something similar eg: stop breaks when webex or google meeting is running, unfortunately, these apps don't spawn unique programs that could be defined in appExclusions, however, we could extend appExclusionsManager.js to start a user-defined program custom-script.sh that would return RC0 when breaks are allowed and RC1 when breaks are disabled using https://nodejs.org/api/child_process.html#child_process_child_process_exec_command_options_callback

"appExclusions": [
    {
        "rule": "pause",
        "active": true,
        "commands": [
            "/usr/share/skypeforlinux/skypeforlinux",
            "atom"
        ]
       "check":"/home/user/custom-script.sh"
    }
]   

Would you agree with this approach?

[brandonsturgeon]

It would be nice (and perhaps more targeted) if this could be extended to screen sharing.

If I'm talking on camera, a break popping up is a nuisance. If I'm sharing something on my screen, it can be very disruptive, especially if strict mode is enabled.

[hovancik]

I wanted something similar eg: stop breaks when webex or google meeting is running, unfortunately, these apps don't spawn unique programs that could be defined in appExclusions, however, we could extend appExclusionsManager.js to start a user-defined program custom-script.sh that would return RC0 when breaks are allowed and RC1 when breaks are disabled using https://nodejs.org/api/child_process.html#child_process_child_process_exec_command_options_callback

"appExclusions": [
    {
        "rule": "pause",
        "active": true,
        "commands": [
            "/usr/share/skypeforlinux/skypeforlinux",
            "atom"
        ]
       "check":"/home/user/custom-script.sh"
    }
]   

Would you agree with this approach?

@vvvlc I am not sure if I understand. Can you please describe it with some specific usecase?

[vvvlc]

I want to disable breaks when google meet or webex meeting is in progress. Unfortunately these meeting apps always run a program regardless meeting is in progress or not. Therefore I cannot use existing "appExclusions" list, because breaks would be always disabled. Therefore I would suggest to run a custom script ahead a break. When script returns RC:0 then break is allowed else break is eg. skipped. until next break. Custom script could detect if a camera is turned on or based on calendar or an active tcpi connection. Code of custom script would be up to user. V

ne 28. 11. 2021 v 12:05 odesílatel Jan Hovancik @.***> napsal:

I wanted something similar eg: stop breaks when webex or google meeting is running, unfortunately, these apps don't spawn unique programs that could be defined in appExclusions, however, we could extend appExclusionsManager.js to start a user-defined program custom-script.sh that would return RC0 when breaks are allowed and RC1 when breaks are disabled using https://nodejs.org/api/child_process.html#child_process_child_process_exec_command_options_callback

"appExclusions": [ { "rule": "pause", "active": true, "commands": [ "/usr/share/skypeforlinux/skypeforlinux", "atom" ] "check":"/home/user/custom-script.sh" } ]

Would you agree with this approach?

@vvvlc https://github.com/vvvlc I am not sure if I understand. Can you please describe it with some specific usecase?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hovancik/stretchly/issues/969#issuecomment-981065041, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADY6XIF3DNIIVZXF56CJVY3UOIEGFANCNFSM46BMFOKQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- V.

[hovancik]

@vvvlc Stretchly has command line interface, see stretchly help so if you have script that detects camera or calendar, you can easily pause Stretchly :)

[lex3001]

I too would love to have Stretchly auto-pause when I am in a Google Meet (on Windows in Chrome). I have so many meetings I have now trained myself to immediately dismiss Stretchly when it pops up in a meeting, so now when it pops up and I am not in a meeting I dismiss it out of habit. I agree that Google Meet should be telling Windows it is in DND mode. Especially if I am actively presenting / screen sharing but also if I am just in the meeting without sharing my screen. But it doesn't. Grr. Has anyone come up with a solution? I suppose some program running as a service/daemon to monitor camera use and/or screen recording could put Windows in and out of DND mode?

[hovancik]

Some sort of browser extension might be the best way to detect if browser is doing some specific thing

[MissCellaneous]

Just wanted to add my voice to this to say that detecting the camera in use / popular video conferencing apps being 'in a call' would be an amazing feature.

I love Stretchly, so thank you :) but when on a Teams call on Windows 10, for example, it will still pop up unless I've remembered to pause it. Even a calendar integration would help in many cases, for planned meetings. App Exclusions don't seem to work for me, perhaps I've added the wrong command, but in any case I expect it wouldn't work for what I need, because technically these apps are always running whether in call or not. Unfortunately Windows Focus Assist is no good to help with this as it's very 'all or nothing' (Microsoft's implementation makes it nothing but a hindrance in my view).

[e-t-l]

Okay, it looks like detecting camera usage is possible, but there's probably not an OS-agnostic method of doing so. There would have to be separate implementations for Windows, Mac, and (probably) Linux.

I'm not very familiar with Macs, but @hovancik would this work? morajabi/electron-is-camera-on (it also looks limited to the built-in mac camera, as opposed to any camera including USB ones).

For Windows, I'm working on a script that will monitor certain Registry key values: If Computer\HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\CapabilityAccessManager\ConsentStore\webcam\*\LastUsedTimeStop = 0x00000000 (0) (where the asterisk indicates all child keys in the \webcam\ key) then that means the camera is currently in use by some program. These keys could be monitored by CMD or PS recursively, perhaps on a 1 minute interval, and send Stretchly a pause command if it finds a value that matches, and unpause when it no longer does.

For Linux, this may be helpful: https://unix.stackexchange.com/questions/344454/how-to-know-if-my-webcam-is-used-or-not

The main limitation I can see of such implementations is that they literally only monitor when the camera is in use, so if you're in a browser-based video call and you've turned off your video, Stretchly wouldn't know that you still want to see everyone else's video. But that could be opened as a separate issue.

[e-t-l]

If we do want one pure-Javascript implementation, I think what we'd have to do is make Stretchly try to access the camera, and if it cannot, that means the camera is in use (or there is no camera - not sure how we'd differentiate that).

https://stackoverflow.com/a/57915667/9621742 This answer has some sample code to do that in a browser, so by my (very limited) understanding it should work in an Electron app too...

[RobinFrcd]

This camera usage detection feature would be really great ! Is it still in the plans ? 🙏

[hovancik]

If someone finds a way how to do it, then we shall implement it :) I've tried links from some of the previous comments but that did not work, One can detect if Stretchly itself runs a camera, but not some another app. Multiple apps could run camera at the same time as well.

[novascreen]

wouldn't it make more sense to detect microphone usage over camera usage as an indicator for being in a meeting? people don't always turn on their video and things like Slack huddles are mostly audio only

[e-t-l]

It's pretty common, especially in larger group meetings, for someone to have video on but mic muted when they're not talking. And the whole point of pausing Stretchly is that it prevents visual interference with a video call. If you're doing an audio-only call like Slack Huddle, why would you need to pause Stretchly at all?

(That being said, I don't think it's a bad idea to provide an option to pause Stretchly when mic is active, in addition to camera. But this specific issue is on the topic of camera use; it might be best to open a new issue if you'd like to suggest an enhancement for mic use. I also don't know how to detect mic use programmatically, so you'd have to do your own research on that.)

[novascreen]

I might still be looking at a document while talking or use screen share. Muting your mic in an app, it might still be technically "in use". Me sharing my video seems like the least relevant signal. I see your point on making a separate ticket though 👍

[novascreen]

I might try https://objective-see.org/products/oversight.html for detection and write a script using the stretchly CLI, i'll report back.

Update: With Oversight I run this script on MacOS

#!/bin/bash

if [[ "$*" == *"-device microphone -event off"* ]]
then
    /Applications/Stretchly.app/Contents/MacOS/Stretchly resume
fi

if [[ "$*" == *"-device microphone -event on"* ]]
then
    /Applications/Stretchly.app/Contents/MacOS/Stretchly pause
fi

you can replace "microphone" with "camera"

[ghost]

Version

• [X] I'm using version 1.7.0

Existing requests

• [X] I've checked Existing requests

What operating system are you using?

Windows

Operating System Version

Windows 10 v 20H2

Problem description

In addition to pausing Stretchly while a specific app is running, there are times users would want to pause Stretchly when accessing certain websites (e.g. browser-based video calling like Google Meet, Jitsi Meet, Zoom (browser), or Facebook Video Call).

Possible solution

While I don't believe there's any way for a JS app to detect the webpages open in a different browser*, there are programmatic ways to detect when the device's camera is in use. This StackOverflow question might be a good starting place.

Alternative solutions

*I could be wrong about detecting open tabs in other browsers. In Windows, for example, when a user hovers over an open browser's taskbar icon, it displays a preview with the name of the active tab. If there's a way to fetch this via Javascript, then we could implement per-website Stretchly exclusions, which would be even better.

Code of Conduct