webcam-capture
webcam-capture copied to clipboard
sarxos
macOS Catalina Issue
My javaFX app uses this library to capture video from a webcam on windows and mac. It has worked great on both platforms up until now. I am testing on macOS Catalina beta2 and now getWebCams() is returning nothing. My discoveryListener is not firing either. I have enabled debug logging as described in the wiki and I don't see any errors. I am using the default driver. Any idea what could be going on??
Hi @burniejm seems there is a change in security permissions in macOS Cataline as: https://developer.apple.com/videos/play/wwdc2019/701/ I am also looking for a solution.
I am actually only using webcam-capture to enumerate cameras and populate a dropdown of cameras. I am then using openCV to grab frames from the selected device index (gathered from webcam-capture). Even hardcoding the device index didn't work until I updated to the newest javaCV (1.5). My project already uses ffmpeg for various video tasks, so I implemented the ffmpeg driver for webcam-capture and used the "list-devices" command to populate my dropdown. Hope this helps.
hi @burniejm may I ask how you are able to run ffmpeg driver on macOS? it expects to have /dev/videoX (X is a . number) device on computer.
hi @burniejm may I ask how you are able to run ffmpeg driver on macOS? it expects to have /dev/videoX (X is a . number) device on computer.
Since I don't actually use the webcam library to capture all I really need is the index of the video devices and the name. I do rely on this library to scan for hardware changes (usb camera add/remove) using the WebcamDiscoveryListener functionality. Here is my pared down getUnixDevices method:
private List<WebcamDevice> getUnixDevices() {
List<WebcamDevice> devices = new ArrayList<WebcamDevice>();
String[] command = {
getCommand(),
"-f",
"avfoundation",
"-list_devices",
"true",
"-hide_banner",
"-i",
"\"\""
};
ProcessBuilder processBuilder = new ProcessBuilder(command);
processBuilder.redirectErrorStream(true);
try {
Process process = processBuilder.start();
InputStream inputStream = process.getInputStream();
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String videoSeperator = "video devices";
String audioSeperator = "audio devices";
String captureScreenDevice = "capture screen";
String line;
while ((line = bufferedReader.readLine()) != null) {
if(line.toLowerCase().contains(videoSeperator)) {
continue;
}
if(line.toLowerCase().contains(captureScreenDevice)) {
break;
}
if(line.toLowerCase().contains(audioSeperator)) {
break;
}
String[] lineArray = line.split("] ");
FFmpegCliDevice device = new FFmpegCliDevice(lineArray[2]);
devices.add(device);
}
try {
process.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
}
} catch (IOException ex) {
ex.printStackTrace();
}
return devices;
}
I added an FFMpegCliDevice constructor that just takes a name as well.
Hi @burniejm thanks for the sharing, definitely only solution we have now to get the camera list and names on Catalina. regards
I am trying to run it on macos catalina beta, I have 2 questions:
1-) It cannot find dependency in pom.xml for webcam-capture-driver-ffmpeg-cli
<dependency> <groupId>com.github.sarxos</groupId> <artifactId>webcam-capture-driver-ffmpeg-cli</artifactId> <version>0.3.12</version> </dependency>
2-) I have tried other drivers listed on the page but, It could not detect any camera with other drivers? How you have resolved it?
@ademzumbul I just manually added the driver files to my project as I couldn't get it to work using gradle.
Hello,
I am using OpenIMAJGrabber on MAC for handling camera device. Now after update MAC OSX to 10.15, i can't get listed devices (empty).
Can anyone share any information about how to work OpenIMAJGrabber on MAC 10.15?
Regards,
Pai
https://github.com/openimaj/openimaj/issues/170
Here is the issue for the default driver used by this library.
I'm having the same issue on Catalina. Looks like work was started on a fix for openimaj/openimaj#170 but no activity since Oct 2019.
The author of openimaj has just pushed an update 1.3.10 that fixes the Catalina issue. @sarxos could you incorporate this update?
I have PR (https://github.com/sarxos/webcam-capture/pull/765) that may address this issue.
Hi guys,
This should already be fixed by commit https://github.com/sarxos/webcam-capture/commit/ae28f268f081f2a1b13f06578a0f4fddb6ffc568 from @kkieffer. Can you please verify this issue with latest JAR, that is 0.3.13-SNAPSHOT, which is available here:
https://oss.sonatype.org/service/local/artifact/maven/redirect?r=snapshots&g=com.github.sarxos&a=webcam-capture&v=0.3.13-SNAPSHOT
@sarxos
I got the following error message on my macOS Catalina 10.15.4.
14:19:13.532 [main] DEBUG com.github.sarxos.webcam.Webcam - Setting new capture driver OpenImajDriver
Exception in thread "main" com.github.sarxos.webcam.WebcamException: java.util.concurrent.ExecutionException: com.github.sarxos.webcam.WebcamException: Cannot execute task
at com.github.sarxos.webcam.WebcamDiscoveryService.getWebcams(WebcamDiscoveryService.java:124)
at com.github.sarxos.webcam.Webcam.getWebcams(Webcam.java:893)
at com.github.sarxos.webcam.Webcam.getDefault(Webcam.java:956)
at com.github.sarxos.webcam.Webcam.getDefault(Webcam.java:933)
at com.github.sarxos.webcam.Webcam.getDefault(Webcam.java:911)
at gist.WebcamPanelExample.main(WebcamPanelExample.java:18)
Caused by: java.util.concurrent.ExecutionException: com.github.sarxos.webcam.WebcamException: Cannot execute task
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at com.github.sarxos.webcam.WebcamDiscoveryService.getWebcams(WebcamDiscoveryService.java:116)
... 5 more
Caused by: com.github.sarxos.webcam.WebcamException: Cannot execute task
at com.github.sarxos.webcam.WebcamProcessor$AtomicProcessor.process(WebcamProcessor.java:72)
at com.github.sarxos.webcam.WebcamProcessor.process(WebcamProcessor.java:140)
at com.github.sarxos.webcam.WebcamTask.process(WebcamTask.java:46)
at com.github.sarxos.webcam.ds.openimaj.OpenImajDriver$GetDevicesTask.getDevices(OpenImajDriver.java:47)
at com.github.sarxos.webcam.ds.openimaj.OpenImajDriver.getDevices(OpenImajDriver.java:67)
at com.github.sarxos.webcam.WebcamDiscoveryService$WebcamsDiscovery.call(WebcamDiscoveryService.java:36)
at com.github.sarxos.webcam.WebcamDiscoveryService$WebcamsDiscovery.call(WebcamDiscoveryService.java:26)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.lang.UnsatisfiedLinkError: 'org.bridj.Pointer org.openimaj.video.capture.OpenIMAJGrabber.getVideoDevices()'
at org.openimaj.video.capture.OpenIMAJGrabber.getVideoDevices(Native Method)
at org.openimaj.video.capture.VideoCapture.getVideoDevices(VideoCapture.java:221)
at com.github.sarxos.webcam.ds.openimaj.OpenImajDriver$GetDevicesTask.handle(OpenImajDriver.java:58)
at com.github.sarxos.webcam.WebcamProcessor$AtomicProcessor.run(WebcamProcessor.java:81)
... 3 more
Process finished with exit code 1
Here is the source code which can be used to reproduce the case.
import javax.swing.JFrame;
import com.github.sarxos.webcam.Webcam;
import com.github.sarxos.webcam.WebcamPanel;
import com.github.sarxos.webcam.WebcamResolution;
import com.github.sarxos.webcam.ds.openimaj.OpenImajDriver;
public class WebcamPanelExample {
static {
Webcam.setDriver(new OpenImajDriver());
}
public static void main(String[] args) throws InterruptedException {
Webcam webcam = Webcam.getDefault();
webcam.setViewSize(WebcamResolution.VGA.getSize());
WebcamPanel panel = new WebcamPanel(webcam);
panel.setFPSDisplayed(true);
panel.setDisplayDebugInfo(true);
panel.setImageSizeDisplayed(true);
panel.setMirrored(true);
JFrame window = new JFrame("Test webcam panel");
window.add(panel);
window.setResizable(true);
window.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
window.pack();
window.setVisible(true);
}
}
Here is the project depenencies in maven approach, and I also tried the Grade way, still NOT work:
<dependency>
<groupId>com.github.sarxos</groupId>
<artifactId>webcam-capture</artifactId>
<version>0.3.13-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.github.sarxos</groupId>
<artifactId>webcam-capture-driver-openimaj</artifactId>
<version>0.3.13-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.openimaj</groupId>
<artifactId>core-video-capture</artifactId>
<version>1.4-SNAPSHOT</version>
</dependency>
IMO. I think this error is caused by the openIMAJ 3rd library, because I got the same error message
Caused by: java.lang.UnsatisfiedLinkError: 'org.bridj.Pointer org.openimaj.video.capture.OpenIMAJGrabber.getVideoDevices()'
when I test the code which only depended on the openIMAJ library without import the webcam-capture package.
So, in your opinion, is there any solution to this problem. Or this is caused by the macOS Catalina 10.15.4, which is just released a few days ago.
@jingmingcn try commenting out this line:
Webcam.setDriver(new OpenImajDriver());
The fix from my PR addressed the built in default driver (which is openImaj) in webcam capture. I neglected to remember that you can also load the external OpenImaj driver (separate source).
@sarxos can you also patch the driver-openimaj under webcam-capture-drivers?
@kkieffer I got the same error message after removing the driver setting command line.
An update about this issue is the case works very well on my macOS Mojave 10.14.6. So I think this issue is caused by the lastest macOS Catalina 10.15.4, which requires more security permission.
I hope the information above will help you to figure out the problem and find a solution. Thanks for your great work.
Can you open your Console.app in /Applications/Utilities and see if anything is logged when you run?
Although I can now see which webcam devices are available, there's a problem when trying to open the camera. In console I see this:
Prompting policy for hardened runtime; service: kTCCServiceCamera requires entitlement com.apple.security.device.camera but it is missing for REQ:{ID: net.java.openjdk.cmd, PID[1631], auid: 501, euid: 501, binary path: '/Library/Frameworks/Java/jdk-14.jdk/Contents/Home/bin/java'}
@kkieffer I got the following message bundling to the process syspolicyd from the console app on the macOS Catalina 10.15.4.
assessment denied for libOpenIMAJGrabber.dylib
com.apple.message.domain: com.apple.security.assessment.outcome2
com.apple.message.signature2: bundle:UNBUNDLED
com.apple.message.signature3: libOpenIMAJGrabber.dylib
com.apple.message.signature5: UNKNOWN
com.apple.message.signature4: 1
com.apple.message.signature: denied:no usable signature
SenderMachUUID: 821D8C33-5B0B-317C-B2CB-29137C319A18
I am not sure whether the updates above would help or not. I am kind of lacking the system driver's developing experience.
Ok - I have a solution, although it's not going to be pretty. But it does work. First, the process "java" does not have an entitlement to use the camera. The only way around this as far as I can figure is to re-sign the java executable. So for this you're going to need a Apple Developer certificate.
-
Find the java binary in your sdk. Mine is located at: /Library/Java/JavaVirtualMachines/current/Contents/Home/bin>
-
Create a file called entitlements.plist in the bin directory. It's the same as the existing java entitlements but I added the camera entitlement at the end.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.cs.allow-jit</key>
<true/>
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
<true/>
<key>com.apple.security.cs.disable-library-validation</key>
<true/>
<key>com.apple.security.cs.allow-dyld-environment-variables</key>
<true/>
<key>com.apple.security.cs.debugger</key>
<true/>
<key>com.apple.security.device.camera</key>
<true/>
</dict>
</plist>
-
Codesign the java binary. Force will re-sign over the existing signature. Note this assumes you have a 3rd party developer certificate in your keychain:
codesign -s "3rd" --force --options runtime --entitlements entitlements.plist java -
Find the Info.plist file located two directories up and copy it into the bin directory:
cp ../../Info.plist . -
Edit this file and add this key:
<key>NSCameraUsageDescription</key>
<string>Arggh Catalina and Java!</string>
Now, assuming you run your test code with this java binary, it should work. You will get a prompt asking for permission to use the camera.
Unfortunately, you're not going to be able to redistribute your jar to anyone with Catalina since they won't have the re-signed java binary.
The solution will be to package into a .app with the correct entitlements. However the JDK14 jpackage tool is broken at the moment and I'm waiting on this fix:
https://bugs.openjdk.java.net/browse/JDK-8237490
@kkieffer Thanks for your solution which I believe will work. But in my own case, the java command is located in the system level folder named /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands, which protected by the macOS SIP. For now, I have two options, one is turning off the SIP which is too risky for my macOS Catalina 10.15.4 laptop carrying important work in progress; another one would be installing a duplicate Java or JDK application to do the compiling work. I will give you the update once I finish the test case.
Thanks for your great idea which really blows my mind.
@kkieffer Here is the update about the solution you provided in the last thread been applied on the macOS Catalina 10.15.4.
First, I have to correct one mistake in the previous thread I posted, which said about the java command in the folder /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands. It turns out that the java located in that folder is not a real java function file, but a java verifying function file. After I unzip a JDK version 14 distribution into the folder /Library/Java/JavaVirtualMachines, the java -version command running in the terminal (or the iTerm2) can detect the last JDK version I just installed.
The conclusion is I have failed to make the solution you have given works on my case. By codesign the java file with the entitlement file and vi the Info.list with the insertion of the camera access declaration, the same error message still comes all the time.
12:28:49.212 [main] INFO com.github.sarxos.webcam.Webcam - WebcamDefaultDriver capture driver will be used
12:28:49.221 [webcam-discovery-service] DEBUG c.g.s.w.d.b.WebcamDefaultDriver - Searching devices
Exception in thread "main" com.github.sarxos.webcam.WebcamException: java.util.concurrent.ExecutionException: com.github.sarxos.webcam.WebcamException: Cannot execute task
at com.github.sarxos.webcam.WebcamDiscoveryService.getWebcams(WebcamDiscoveryService.java:124)
at com.github.sarxos.webcam.Webcam.getWebcams(Webcam.java:893)
at com.github.sarxos.webcam.Webcam.getDefault(Webcam.java:956)
at com.github.sarxos.webcam.Webcam.getDefault(Webcam.java:933)
at com.github.sarxos.webcam.Webcam.getDefault(Webcam.java:911)
at gist.WebcamPanelExample.main(WebcamPanelExample.java:18)
Caused by: java.util.concurrent.ExecutionException: com.github.sarxos.webcam.WebcamException: Cannot execute task
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at com.github.sarxos.webcam.WebcamDiscoveryService.getWebcams(WebcamDiscoveryService.java:116)
... 5 more
Caused by: com.github.sarxos.webcam.WebcamException: Cannot execute task
at com.github.sarxos.webcam.WebcamProcessor$AtomicProcessor.process(WebcamProcessor.java:72)
at com.github.sarxos.webcam.WebcamProcessor.process(WebcamProcessor.java:140)
at com.github.sarxos.webcam.WebcamTask.process(WebcamTask.java:46)
at com.github.sarxos.webcam.ds.buildin.WebcamDefaultDriver$GetDevicesTask.getDevices(WebcamDefaultDriver.java:79)
at com.github.sarxos.webcam.ds.buildin.WebcamDefaultDriver.getDevices(WebcamDefaultDriver.java:124)
at com.github.sarxos.webcam.WebcamDiscoveryService$WebcamsDiscovery.call(WebcamDiscoveryService.java:36)
at com.github.sarxos.webcam.WebcamDiscoveryService$WebcamsDiscovery.call(WebcamDiscoveryService.java:26)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.UnsatisfiedLinkError: 'org.bridj.Pointer com.github.sarxos.webcam.ds.buildin.natives.OpenIMAJGrabber.getVideoDevices()'
at com.github.sarxos.webcam.ds.buildin.natives.OpenIMAJGrabber.getVideoDevices(Native Method)
at com.github.sarxos.webcam.ds.buildin.WebcamDefaultDriver$GetDevicesTask.handle(WebcamDefaultDriver.java:93)
at com.github.sarxos.webcam.WebcamProcessor$AtomicProcessor.run(WebcamProcessor.java:81)
... 3 more
Process finished with exit code 1
assessment denied for libOpenIMAJGrabber.dylib
com.apple.message.domain: com.apple.security.assessment.outcome2
com.apple.message.signature2: bundle:UNBUNDLED
com.apple.message.signature3: libOpenIMAJGrabber.dylib
com.apple.message.signature5: UNKNOWN
com.apple.message.signature4: 1
com.apple.message.signature: denied:no usable signature
SenderMachUUID: 821D8C33-5B0B-317C-B2CB-29137C319A18
Thanks for your help anyway. Maybe the cause of the problem is due to my individual special case on macOS Catalina 10.15.4. I think I should switch to the macOS Mojave laptop to continue the work until the fix released in the future. So may God bless the programmers working on the openIMAJ project.
Here are some supplementary notes. When doing the codesign step, I used the identity belongs to my personal developer account, which is free and not officially confirmed by the Apple Developer Center. I think this may be the reason why I failed to apply the solution.
@kkieffer Here is the update.
I found a solution to solve this issue, which is by explicitly declaring an environment variable name DYLD_LIBRARY_PATH to a folder contains the libOpenIMAJGrabber.dylib file.
Sorry for wasting your valuable time on a stupid question I have posted.
Hi @jingmingcn, @kkieffer,
I believe this is not a stupid question at all. It's a real issue. Adding libOpenIMAJGrabber.dylib to DYLD_LIBRARY_PATH environment variable is only a workaround. Ideally I would like this dylib to be loaded automatically the same way as it's done for dll on Windows and so on Linux.
I'm not really familiar with the MacOS permissions, especially on the newer systems, and with current COVID-19 situation, have no access to Mac machine where I can test. If you have any ideas on how we can permanently fix this issue I'm willing to work this out somehow. Can building and signing it properly help in this case?
Ah, it turns out I had a copy of libOpenIMAJGrabber.dylib in my /usr/local/lib directory, which is why it worked for me. Removing this, I see:
Exception in thread "main" java.lang.UnsatisfiedLinkError: 'org.bridj.Pointer org.openimaj.video.capture.OpenIMAJGrabber.getVideoDevices()
I'm not familiar with bridj so I'll have to dig into it again. Perhaps @jonhare could comment.
@kkieffer
I have two suspicions:
Option 1, a wrong BridJ version. Now when I look at the POM there is BridJ 0.7.0 used in Webcam Capture API and 0.7-20140918-2 in OpenIMAJ. To verify this case you can download version 0.7-20140918-2 from OpenIMAJ Maven repo:
http://maven.ecs.soton.ac.uk/content/groups/maven.openimaj.org/com/nativelibs4java/bridj/0.7-20140918-2/bridj-0.7-20140918-2.jar
Replace original BridJ with the one you downloaded and redo the test. If it fails, move to option 2.
Option 2, and I only guess, may be caused by the denied assessment mentioned in https://github.com/sarxos/webcam-capture/issues/723#issuecomment-606679227. If this is the case then maybe having your version of signed dylib would be a better option. At least this is something we could test against to verify if this is the root cause.
Option 1 - That works! However I had to manually do this, trying to change it in the pom file, I get this error:
The POM for com.nativelibs4java:bridj:jar:0.7-20140918-2 is missing, no dependency information available. Failed to execute goal on project webcam-capture: Could not resolve dependencies for project com.github.sarxos:webcam-capture:bundle:0.3.13-SNAPSHOT: Failure to find com.nativelibs4java:bridj:jar:0.7-20140918-2 in https://repo.maven.apache.org/maven2
I guess the folks from bridj need put release their next version.
@kkieffer @sarxos I remember now - there was a problem with dyncall in the bridj 0.7 (and 0.7-20140918) builds that meant that it wouldn't pick up libraries loaded from unusual locations (bridj stores the libs in the jar, and then unpacks them and links them dynamically) when using Catalina. This was in addition to the deprecated video capture apis that I had to rewrite the code for. My modified bridj is in the openimaj maven repo (just add maven.openimaj.org as an additional repo in your pom file and it should work): http://maven.ecs.soton.ac.uk/content/groups/maven.openimaj.org/com/nativelibs4java/bridj/0.7-20140918-2/ All it does is update the dyncall version (from last October).
I suspect that going forwards we are going to hit problems with the dylibs not being signed though that will need addressing. I can probably sign the libOpenIMAJGrabber.dylib and the bridj dylibs, but I'm not sure if that will be enough (the java executable would need signing and also need entitlements to load dylibs from arbitrary places; I'm guessing the bridj dylibs need similar entitlements and the OpenIMAJ one needs the camera entitlement). I literally know nothing about any of this though!!
@jonhare can you submit your fix to bridj to their mainline?
I have not had a problem using unsigned openimaj library at this point but I'm actually surprised that it works without signing. In addition to signing the dylibs must be notarized for Catalina.
I don't think the dylibs need any entitlements. That's for the binary (java) to supply.