javacpp-presets
javacpp-presets copied to clipboard
bytedeco
Possible to do a headless packaging of OpenCV and FFmpeg?
I'm running on a system with no GTK which is needed by highgui (and probably others). However, I have no need to run any GUI and getting GTK installed on the target system is at present, not possible.
Exception in thread "main" java.lang.UnsatisfiedLinkError: no jniopencv_highgui in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1225)
at org.bytedeco.javacpp.Loader.load(Loader.java:983)
at org.bytedeco.javacpp.Loader.load(Loader.java:882)
at org.bytedeco.javacpp.opencv_highgui.<clinit>(opencv_highgui.java:15)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.bytedeco.javacpp.Loader.load(Loader.java:941)
at org.bytedeco.javacpp.Loader.load(Loader.java:898)
...
Caused by: java.lang.UnsatisfiedLinkError: some.jar/org/bytedeco/javacpp/linux-x86_64/libjniopencv_highgui.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1205)
...
When I remove all references to highgui from the generated jar, I predictably get a different error:
Exception in thread "main" java.lang.ArrayStoreException: sun.reflect.annotation.TypeNotPresentExceptionProxy
at sun.reflect.annotation.AnnotationParser.parseClassArray(AnnotationParser.java:724)
at sun.reflect.annotation.AnnotationParser.parseArray(AnnotationParser.java:531)
at sun.reflect.annotation.AnnotationParser.parseMemberValue(AnnotationParser.java:355)
at sun.reflect.annotation.AnnotationParser.parseAnnotation2(AnnotationParser.java:286)
at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:120)
at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:72)
at java.lang.Class.createAnnotationData(Class.java:3521)
at java.lang.Class.annotationData(Class.java:3510)
at java.lang.Class.createAnnotationData(Class.java:3526)
at java.lang.Class.annotationData(Class.java:3510)
at java.lang.Class.getAnnotation(Class.java:3415)
at org.bytedeco.javacpp.Loader.checkPlatform(Loader.java:816)
at org.bytedeco.javacpp.Loader.load(Loader.java:918)
at org.bytedeco.javacpp.Loader.load(Loader.java:898)
Sure, it's possible to do that. Modify the cppbuild.sh script and follow the build instructions: https://github.com/bytedeco/javacpp-presets#build-instructions
BTW, it won't actually try to display anything by default anyway. It just needs to link with GTK, so we can just install those libraries on the system and it will work just fine without display.
The issue isn’t that I’m afraid it will display anything. My issue is I have no control over the target system so I can’t install them.
Libraries can still be installed locally. Try to put some random libgtk-x11-2.0.so.0 in your library path and it should work. Even some dummy library with nothing in it should work.
For context, this is being deployed to AWS Lambda. The decision to use Lambda is out of my hands (and certainly isn't my choice for now) but it is increasingly common and likely to become increasingly problematic for people using the presets.
I saw a previous issue that suggested publishing (to e.g. Maven) a pre-built headless version. Is that something that is still being considered?
Alternatively, I also saw mention of bundling in libgtk and friends (as is done with libgomp) was also considered at one point. Is that something that is still being considered?
As an aside, just dealing with libgtk isn't nearly enough:
libatk-1.0.so.0 => not found
libcairo.so.2 => not found
libfontconfig.so.1 => not found
libfreetype.so.6 => not found
libgdk-x11-2.0.so.0 => not found
libgdk_pixbuf-2.0.so.0 => not found
libgomp.so.1 => not found
libgtk-x11-2.0.so.0 => not found
libpango-1.0.so.0 => not found
libpangocairo-1.0.so.0 => not found
libpangoft2-1.0.so.0 => not found
Yeah, it looks like opencv_highgui links with everything. Well, that's an issue with OpenCV. Modules shouldn't link with opencv_highgui if they don't need to display something, so if it's something you feel is really important, you should also report upstream...
This helped, me, for those that come across my initial question with docker, you have to install one of these lib deps. I am figuring out the least required dep path for my personal use. But nice work guys. Thanks for the reference @saudet this solved my issue.
@KennyBarraud You need to make sure the binaries for the native platforms are also bundled.
I'm using Scala, I've trouble deploying javacv in a docker using sbt DockerPlugin , is there a way to get the headless build without manually building the javaccp.
not sure why it gets the library from non existent folder "/home/{USER}/.javacpp/cache/" and using jniopencv_highgui , I believe is due to linking with all the other opencv libraries.
Exception in thread "io-compute-2" java.lang.UnsatisfiedLinkError: no jniopencv_highgui in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
at java.lang.Runtime.loadLibrary0(Runtime.java:871)
at java.lang.System.loadLibrary(System.java:1124)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1825)
at org.bytedeco.javacpp.Loader.load(Loader.java:1416)
at org.bytedeco.javacpp.Loader.load(Loader.java:1227)
at org.bytedeco.javacpp.Loader.load(Loader.java:1203)
at org.bytedeco.opencv.global.opencv_highgui.<clinit>(opencv_highgui.java:23)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.bytedeco.javacpp.Loader.load(Loader.java:1282)
at org.bytedeco.javacpp.Loader.load(Loader.java:1227)
at org.bytedeco.javacpp.Loader.load(Loader.java:1203)
at org.bytedeco.opencv.opencv_text.IntVector.<clinit>(IntVector.java:35)
at example.backend.vision.DetectorOpenCV.postprocess(DetectorOpenCV.scala:82)
at example.backend.vision.DetectorOpenCV.get(DetectorOpenCV.scala:182)
at example.backend.vision.FrameReader$.testDetect$1(FrameReader.scala:81)
at example.backend.vision.FrameReader$.$anonfun$testRun$2(FrameReader.scala:89)
at example.backend.vision.FrameReader$.$anonfun$testRun$2$adapted(FrameReader.scala:89)
at example.backend.vision.FrameReader.$anonfun$runProcess$3(FrameReader.scala:36)
at example.backend.vision.FrameReader.$anonfun$runProcess$3$adapted(FrameReader.scala:35)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1279)
at example.backend.vision.FrameReader.runProcess(FrameReader.scala:35)
at example.backend.vision.FrameReader$.testRun(FrameReader.scala:89)
at example.backend.Routes$$anonfun$routes$1.applyOrElse(Routes.scala:31)
at example.backend.Routes$$anonfun$routes$1.applyOrElse(Routes.scala:27)
at scala.PartialFunction$Lifted.apply(PartialFunction.scala:313)
at scala.PartialFunction$Lifted.apply(PartialFunction.scala:309)
at org.http4s.HttpRoutes$.$anonfun$of$2(HttpRoutes.scala:80)
at cats.syntax.FlatMapOps$.$anonfun$$greater$greater$1(flatMap.scala:54)
at cats.effect.IOFiber.next$2(IOFiber.scala:398)
at cats.effect.IOFiber.runLoop(IOFiber.scala:409)
at cats.effect.IOFiber.execR(IOFiber.scala:1332)
at cats.effect.IOFiber.run(IOFiber.scala:139)
at cats.effect.unsafe.WorkerThread.run(WorkerThread.scala:461)
Caused by: java.lang.UnsatisfiedLinkError: /home/demiourgos728/.javacpp/cache/org.bytedeco.opencv-4.6.0-1.5.8-SNAPSHOT-linux-arm64.jar/org/bytedeco/opencv/linux-arm64/libjniopencv_highgui.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
at java.lang.Runtime.load0(Runtime.java:810)
at java.lang.System.load(System.java:1088)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1772)
... 33 more
java.lang.UnsatisfiedLinkError: no jniopencv_highgui in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
at java.lang.Runtime.loadLibrary0(Runtime.java:871)
at java.lang.System.loadLibrary(System.java:1124)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1825)
at org.bytedeco.javacpp.Loader.load(Loader.java:1416)
at org.bytedeco.javacpp.Loader.load(Loader.java:1227)
at org.bytedeco.javacpp.Loader.load(Loader.java:1203)
at org.bytedeco.opencv.global.opencv_highgui.<clinit>(opencv_highgui.java:23)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.bytedeco.javacpp.Loader.load(Loader.java:1282)
at org.bytedeco.javacpp.Loader.load(Loader.java:1227)
at org.bytedeco.javacpp.Loader.load(Loader.java:1203)
at org.bytedeco.opencv.opencv_text.IntVector.<clinit>(IntVector.java:35)
at example.backend.vision.DetectorOpenCV.postprocess(DetectorOpenCV.scala:82)
at example.backend.vision.DetectorOpenCV.get(DetectorOpenCV.scala:182)
at example.backend.vision.FrameReader$.testDetect$1(FrameReader.scala:81)
at example.backend.vision.FrameReader$.$anonfun$testRun$2(FrameReader.scala:89)
at example.backend.vision.FrameReader$.$anonfun$testRun$2$adapted(FrameReader.scala:89)
at example.backend.vision.FrameReader.$anonfun$runProcess$3(FrameReader.scala:36)
at example.backend.vision.FrameReader.$anonfun$runProcess$3$adapted(FrameReader.scala:35)
at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1279)
at example.backend.vision.FrameReader.runProcess(FrameReader.scala:35)
at example.backend.vision.FrameReader$.testRun(FrameReader.scala:89)
at example.backend.Routes$$anonfun$routes$1.applyOrElse(Routes.scala:31)
at example.backend.Routes$$anonfun$routes$1.applyOrElse(Routes.scala:27)
at scala.PartialFunction$Lifted.apply(PartialFunction.scala:313)
at scala.PartialFunction$Lifted.apply(PartialFunction.scala:309)
at org.http4s.HttpRoutes$.$anonfun$of$2(HttpRoutes.scala:80)
at cats.syntax.FlatMapOps$.$anonfun$$greater$greater$1(flatMap.scala:54)
at cats.effect.IOFiber.next$2(IOFiber.scala:398)
at cats.effect.IOFiber.runLoop(IOFiber.scala:409)
at cats.effect.IOFiber.execR(IOFiber.scala:1332)
at cats.effect.IOFiber.run(IOFiber.scala:139)
at cats.effect.unsafe.WorkerThread.run(WorkerThread.scala:461)
Caused by: java.lang.UnsatisfiedLinkError: /home/demiourgos728/.javacpp/cache/org.bytedeco.opencv-4.6.0-1.5.8-SNAPSHOT-linux-arm64.jar/org/bytedeco/opencv/linux-arm64/libjniopencv_highgui.so: libgtk-x11-2.0.so.0: cannot open shared object file: No such file or directory
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
at java.lang.Runtime.load0(Runtime.java:810)
at java.lang.System.load(System.java:1088)
at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1772)
... 33 more
this is my sbt dependencies.
val javacppVersion = "1.5.8-SNAPSHOT"
val cpuPlatform = "linux-arm64"
val opencv = "4.5.5"
val opencvPlatform = "4.5.5-1.5.7"
val ffmpeg = "5.0"
val ffmpegPlatform = "5.0-1.5.7"
val openblas = "0.3.19"
lazy val presetLibs = Seq(
"opencv" -> V.opencv
).flatMap { case (lib, ver) =>
Seq(
"org.bytedeco" % lib % s"$ver-${Project.javacppVersion}",
"org.bytedeco" % lib % s"$ver-${Project.javacppVersion}" classifier Project.cpuPlatform
)
}
lazy val presetLibs2 = Seq(
"ffmpeg" -> V.ffmpeg,
"openblas" -> V.openblas
//"openblas-platform" -> V.openblas
).flatMap { case (lib, ver) =>
Seq(
"org.bytedeco" % lib % s"$ver-1.5.7",
"org.bytedeco" % lib % s"$ver-1.5.7" classifier Project.cpuPlatform
)
}
lazy val backend: Seq[Def.Setting[_]] = common ++ Seq(
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots",
libraryDependencies ++=
http4sModules.map("org.http4s" %% _ % V.http4s) ++
Seq(
"org.bytedeco" % "javacpp" % Project.javacppVersion,
"org.bytedeco" % "javacpp" % Project.javacppVersion classifier Project.cpuPlatform,
"org.bytedeco" % "javacv" % Project.javacppVersion,
"org.scala-lang.modules" %% "scala-swing" % V.scalaSwing,
"org.scalafx" %% "scalafx" % V.scalafx,
"org.scalafx" %% "scalafx-extras" % V.scalafxExtras,
"org.bytedeco" % "ffmpeg-platform" % V.ffmpegPlatform,
"org.bytedeco" % "opencv-platform" % V.opencvPlatform,
) ++ presetLibs ++ presetLibs2,
dependencyOverrides ++= Seq("org.typelevel" % "cats-effect_2.13" % "3.3.12")
)
help will be very much appreciated!
@thekevshow
This helped, me, for those that come across my initial question with docker, you have to install one of these lib deps. I am figuring out the least required dep path for my personal use. But nice work guys. Thanks for the reference @saudet this solved my issue.
hi, what library dependencies did you install? can you tell me how you work around it? thanks in advance. i have no option to run apt-get install btw.
@thekevshow
This helped, me, for those that come across my initial question with docker, you have to install one of these lib deps. I am figuring out the least required dep path for my personal use. But nice work guys. Thanks for the reference @saudet this solved my issue.
hi, what library dependencies did you install? can you tell me how you work around it? thanks in advance. i have no option to run apt-get install btw.
have you solved this problem? @alawasoft
