spark-deep-learning
spark-deep-learning copied to clipboard
sparkdl for using in Anaconda
Hello all I am wondering if there is any method to install sparkdl within Anaconda and using conda command? I am trying to run a project in ipython using pyspark and the code is using: from sparkdl import readImages everything work well except saprkdl which is not recognized and I could not find any way to install sparkdl for my Anaconda.
Your help is really appreciated Fari
@fsahba making a package for pypi and conda is not easy because the python code calls all sorts of java libraries, which are also embedded. You need to start Spark and IPython using the spark shell (see [1] for example). See this issue on how you can then use the package: https://github.com/databricks/spark-deep-learning/issues/19
[1] https://stackoverflow.com/questions/31862293/how-to-load-ipython-shell-with-pyspark