spark-tfocs icon indicating copy to clipboard operation
spark-tfocs copied to clipboard

Is this library compitable with Apache Spark 2.1.1 ?

Open AM-DNEGEL opened this issue 7 years ago • 1 comments

Using Apache Spark 2.1.1

TFOCS.optimize(new SmoothQuad(observations), new LinopMatrix(designMatrix), new ProxL1(0), x0)

java.lang.NoClassDefFoundError: org/apache/spark/Logging at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at java.net.URLClassLoader.access$100(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:368) at java.net.URLClassLoader$1.run(URLClassLoader.java:362) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:361) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) at java.lang.ClassLoader.loadClass(ClassLoader.java:411) at java.lang.ClassLoader.loadClass(ClassLoader.java:411) at java.lang.ClassLoader.loadClass(ClassLoader.java:411) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 58 elided

AM-DNEGEL avatar Sep 07 '17 15:09 AM-DNEGEL

Hi AM-DNEGEL, TFOCS for Spark was developed for Spark 1.4.1, and some changes may be necessary for compatibility with newer Spark versions. This has been mentioned in https://github.com/databricks/spark-tfocs/issues/26

staple avatar Sep 08 '17 22:09 staple