clojure-mxnet Running on Ubuntu Linux

Running on Ubuntu Linux

Open iarenaza opened this issue 6 years ago • 9 comments

Using the pre-compiled binaries of clojure-mxnet and the pre-compiled binaries of mxnet it depends on doesn't work in Ubuntu 16.04 (and probably any later version as of today). Neither the CPU nor the GPU version.

When you run lein test is errors out with:

magnet@3d847d7fd236:~/app$ lein test
INFO  MXNetJVM: Try loading mxnet-scala from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-gpu from native path.                               
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-cpu from native path.
WARN  MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider ins
talling the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cm
d option -Djava.library.path=[lib path].
INFO  org.apache.mxnet.util.NativeLibraryLoader: Loading libmxnet-scala.so from /lib/native/ copying to mxnet-scala
ERROR org.apache.mxnet.util.NativeLibraryLoader: Couldn't load copied link file: java.lang.UnsatisfiedLinkError: /t
mp/mxnet4601529779960734329/mxnet-scala: libopencv_imgcodecs.so.3.4: cannot open shared object file: No such file o
r directory
ERROR MXNetJVM: Couldn't find native library mxnet-scala
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/mxnet4601529779960734329/mxnet-scala: libopencv_img
codecs.so.3.4: cannot open shared object file: No such file or directory, compiling:(base.clj:4:19)
        at clojure.lang.Compiler$StaticMethodExpr.eval(Compiler.java:1733)
        at clojure.lang.Compiler$DefExpr.eval(Compiler.java:457)
        at clojure.lang.Compiler.eval(Compiler.java:7067)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.invoke(RestFn.java:551)
        at org.apache.clojure_mxnet.io$eval2976$loading__6434__auto____2977.invoke(io.clj:1)
        at org.apache.clojure_mxnet.io$eval2976.invokeStatic(io.clj:1)
        at org.apache.clojure_mxnet.io$eval2976.invoke(io.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7051)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.invoke(RestFn.java:703)
        at org.apache.clojure_mxnet.conv_test$eval351$loading__6434__auto____352.invoke(conv_test.clj:1)
        at org.apache.clojure_mxnet.conv_test$eval351.invokeStatic(conv_test.clj:1)
        at org.apache.clojure_mxnet.conv_test$eval351.invoke(conv_test.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7051)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$apply.invoke(core.clj:652)
        at user$eval233.invokeStatic(form-init1643472251265275304.clj:1)
        at user$eval233.invoke(form-init1643472251265275304.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7052)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.Compiler.loadFile(Compiler.java:7452)
        at clojure.main$load_script.invokeStatic(main.clj:278)
        at clojure.main$init_opt.invokeStatic(main.clj:280)
        at clojure.main$init_opt.invoke(main.clj:280)
        at clojure.main$initialize.invokeStatic(main.clj:311)
        at clojure.main$null_opt.invokeStatic(main.clj:345)
        at clojure.main$null_opt.invoke(main.clj:342)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/mxnet4601529779960734329/mxnet-scala: libopencv_imgcodecs.so.3.4: cannot open shared object file: No such file or directory
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
        at java.lang.Runtime.load0(Runtime.java:809)
        at java.lang.System.load(System.java:1086)
        at org.apache.mxnet.util.NativeLibraryLoader$.loadLibraryFromStream(NativeLibraryLoader.scala:140)
        at org.apache.mxnet.util.NativeLibraryLoader$.loadLibrary(NativeLibraryLoader.scala:93)
        at org.apache.mxnet.Base$.<init>(Base.scala:70)
        at org.apache.mxnet.Base$.<clinit>(Base.scala)
        at org.apache.mxnet.Base.MX_REAL_TYPE(Base.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
        at clojure.lang.Compiler$StaticMethodExpr.eval(Compiler.java:1726)
        ... 100 more
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet4601529779960734329/mxnet-scala
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet4601529779960734329
Tests failed.

The problem is that it can't find libopencv_imgcodecs.so.3.4 shared library. And that library is part of OpenCV 3.4 (the version used to compile MxNet 1.2). But OpenCV 3.4 is not packaged in Ubuntu 16.04 (nor any other later, including 18.04). The latest OpenCV version packaged in Ubuntu as of today is 3.2.

There are PPA packages for OpenCV 3.4 available at https://launchpad.net/~timsc/+archive/ubuntu/opencv-3.4

Once you add the PPA repository, udpate the package list and install the libopencv-imgcodecs3.4 package, it errors out with:

magnet@08376acd332f:~/app$ lein test
INFO  MXNetJVM: Try loading mxnet-scala from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-gpu from native path.
INFO  MXNetJVM: Try loading mxnet-scala-linux-x86_64-cpu from native path.
WARN  MXNetJVM: MXNet Scala native library not found in path. Copying native library from the archive. Consider ins
talling the library somewhere in the path (for Windows: PATH, for Linux: LD_LIBRARY_PATH), or specifying by Java cm
d option -Djava.library.path=[lib path].
INFO  org.apache.mxnet.util.NativeLibraryLoader: Loading libmxnet-scala.so from /lib/native/ copying to mxnet-scala
ERROR org.apache.mxnet.util.NativeLibraryLoader: Couldn't load copied link file: java.lang.UnsatisfiedLinkError: /t
mp/mxnet902995114025761212/mxnet-scala: libcurl.so.4: cannot open shared object file: No such file or directory
ERROR MXNetJVM: Couldn't find native library mxnet-scala
Exception in thread "main" java.lang.UnsatisfiedLinkError: /tmp/mxnet902995114025761212/mxnet-scala: libcurl.so.4:
cannot open shared object file: No such file or directory, compiling:(base.clj:4:19)
        at clojure.lang.Compiler$StaticMethodExpr.eval(Compiler.java:1733)
        at clojure.lang.Compiler$DefExpr.eval(Compiler.java:457)
        at clojure.lang.Compiler.eval(Compiler.java:7067)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.invoke(RestFn.java:551)
        at org.apache.clojure_mxnet.io$eval2976$loading__6434__auto____2977.invoke(io.clj:1)
        at org.apache.clojure_mxnet.io$eval2976.invokeStatic(io.clj:1)
        at org.apache.clojure_mxnet.io$eval2976.invoke(io.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7051)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.invoke(RestFn.java:703)
        at org.apache.clojure_mxnet.conv_test$eval351$loading__6434__auto____352.invoke(conv_test.clj:1)
        at org.apache.clojure_mxnet.conv_test$eval351.invokeStatic(conv_test.clj:1)
        at org.apache.clojure_mxnet.conv_test$eval351.invoke(conv_test.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7051)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.RT.loadResourceScript(RT.java:379)
        at clojure.lang.RT.loadResourceScript(RT.java:370)
        at clojure.lang.RT.load(RT.java:460)
        at clojure.lang.RT.load(RT.java:426)
        at clojure.core$load$fn__6548.invoke(core.clj:6046)
        at clojure.core$load.invokeStatic(core.clj:6045)
        at clojure.core$load.doInvoke(core.clj:6029)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invokeStatic(core.clj:5848)
        at clojure.core$load_one.invoke(core.clj:5843)
        at clojure.core$load_lib$fn__6493.invoke(core.clj:5888)
        at clojure.core$load_lib.invokeStatic(core.clj:5887)
        at clojure.core$load_lib.doInvoke(core.clj:5868)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$load_libs.invokeStatic(core.clj:5925)
        at clojure.core$load_libs.doInvoke(core.clj:5909)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$require.invokeStatic(core.clj:5947)
        at clojure.core$require.doInvoke(core.clj:5947)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invokeStatic(core.clj:659)
        at clojure.core$apply.invoke(core.clj:652)
        at user$eval233.invokeStatic(form-init2605659760700050858.clj:1)
        at user$eval233.invoke(form-init2605659760700050858.clj:1)
        at clojure.lang.Compiler.eval(Compiler.java:7062)
        at clojure.lang.Compiler.eval(Compiler.java:7052)
        at clojure.lang.Compiler.load(Compiler.java:7514)
        at clojure.lang.Compiler.loadFile(Compiler.java:7452)
        at clojure.main$load_script.invokeStatic(main.clj:278)
        at clojure.main$init_opt.invokeStatic(main.clj:280)
        at clojure.main$init_opt.invoke(main.clj:280)
        at clojure.main$initialize.invokeStatic(main.clj:311)
        at clojure.main$null_opt.invokeStatic(main.clj:345)
        at clojure.main$null_opt.invoke(main.clj:342)
        at clojure.main$main.invokeStatic(main.clj:424)
        at clojure.main$main.doInvoke(main.clj:387)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.lang.Var.applyTo(Var.java:702)
        at clojure.main.main(main.java:37)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/mxnet902995114025761212/mxnet-scala: libcurl.so.4: cannot open shared object file: No such file or directory
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
        at java.lang.Runtime.load0(Runtime.java:809)
        at java.lang.System.load(System.java:1086)
        at org.apache.mxnet.util.NativeLibraryLoader$.loadLibraryFromStream(NativeLibraryLoader.scala:140)
        at org.apache.mxnet.util.NativeLibraryLoader$.loadLibrary(NativeLibraryLoader.scala:93)
        at org.apache.mxnet.Base$.<init>(Base.scala:70)
        at org.apache.mxnet.Base$.<clinit>(Base.scala)
        at org.apache.mxnet.Base.MX_REAL_TYPE(Base.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93)
        at clojure.lang.Compiler$StaticMethodExpr.eval(Compiler.java:1726)
        ... 100 more
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet902995114025761212/mxnet-scala
INFO  org.apache.mxnet.util.NativeLibraryLoader: Deleting /tmp/mxnet902995114025761212
Tests failed.

So you also need to install libcurl3 package (already part of Ubuntu). After installing both packages (in addition to those mentionned in the MxNet installation instructions for Ubuntu), all the tests run flawlessly both with the GPU and CPU version.

This is the complete list of packages we (Magnet) are using to run clojure-mxnet in a Docker container, based on nVidia's nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04 base image:

libopenblas-base
libatlas3-base
libcurl3
libopencv-imgcodecs3.4

As mentionned before, you need to add the extra PPA repository to be able to install libopencv-imgcodecs3.4.

We can provide the Dockerfile file we are using, if you think it's helpful to other people.

Greetins, Iñaki.

Jun 07 '18 08:06 iarenaza

Thanks - I expect that plenty of other people will run across this too.

I have to wonder if this, and #1, are both upstream issues with the Scala bindings in MXNet (or with the JARs they've pushed to Maven). As far as I understand it, any native library issue that occurs here should occur likewise with the Scala bindings on their own.

Jun 07 '18 11:06 Hodapp87

@iarenaza Thanks for the update on all that and providing the details, (I missed it earlier on my phone notification). It will really help!

Jun 07 '18 14:06 gigasquid

@iarenaza A Dockerfile will be most useful if you are up for it. Please feel free to open a PR 😸

Jun 07 '18 14:06 gigasquid

@iarenaza I'm planning on getting a PR together for the MXNet main project tomorrow/ this weekend. So maybe you want to hold off on submitting it here, that way you can just get on that PR if you want. I'll ping you here when it is up

Jun 07 '18 17:06 gigasquid

@Hodapp87 @iarenaza I included you both in the Special Thanks section of the README - if you have any edits/changes - please let me know 😸

Jun 08 '18 13:06 gigasquid

@gigasquid I did all this work under contract with Magnet Coop, so they deserve the credit as well :cat:

Jun 08 '18 15:06 iarenaza

@iarenaza Cool - added them too :)

Jun 08 '18 15:06 gigasquid

@iarenaza here is the PR https://github.com/apache/incubator-mxnet/pull/11205

Jun 08 '18 20:06 gigasquid

Thanks @iarenaza. I also run through java.lang.UnsatisfiedLinkError: /tmp/mxnet2725232243239797853/mxnet-scala: libopencv_imgcodecs.so.3.4: cannot open shared object file: No such file or directory with MXNET 1.2.1

Simply adding the ppa for opencv-3.4 fixed the problem in my case

Jul 25 '18 22:07 mdespriee

clojure-mxnet clojure-mxnet copied to clipboard

Running on Ubuntu Linux

clojure-mxnet
clojure-mxnet copied to clipboard