spark-deep-learning
spark-deep-learning copied to clipboard
databricks
Deep Learning Pipelines for Apache Spark
I am running the following code to try to fit a model ``` from sparkdl.xgboost import XgboostClassifier param = { 'num_workers': 4, # number of workers on the cluster, adjust...
Hi, Per [this overflow question](https://stackoverflow.com/questions/69146212/deepimagefeaturizer-returing-error-cannot-import-name-resnet50-from-keras-applic) one needs to modify `/home/user/.local/lib/python3.8/site-packages/sparkdl/transformers/keras_applications.py` . This happened in databricks using v 10.3. Have to change `from keras.applications import inception_v3, xception, resnet50` to `from keras.applications...
I'm using Spark 2.4.2 with Anaconda python 3.6.5. I'm getting below error. What's the best way to resolve this? Command: pyspark --master local[*] --packages databricks:spark-deep-learning:1.5.0-spark2.4-s_2.11 >>> from pyspark.ml.classification import LogisticRegression...
Hi team, I am using Azure Databricks and doing some pipelining using spark and model using keras and tensorflow. Recently I had to update my cluster from 5.4 to 6.2....
https://github.com/hongzimao/decima-sim would you like to help me to integrate this deep learning model into your pipeline> how can I integrate or plug it with your frameworks?
Hi, I'm developing a neural network using Pytorch in a non-databricks cluster to ensure its functionality prior migrating to a databricks cluster. Since I'm using Pytorch, I don't need Keras...
**I find it so many ‘spark-deep-learning’, such as :** elephas：https://github.com/maxpumperla/elephas dist-keras：https://github.com/cerndb/dist-keras sparknet：https://github.com/amplab/sparknet dl4j：https://github.com/deeplearning4j/dl4j-spark-ml TensorFlowOnSpark：https://github.com/yahoo/TensorFlowOnSpark **spark-deep-learning：https://github.com/databricks/spark-deep-learning** H2O：https://github.com/h2oai/sparkling-water/tree/master/ BigDL：https://github.com/intel-analytics/BigDL analytics-zoo：https://github.com/intel-analytics/analytics-zoo **It looks like BigDL is the most active one. I want to...
Here is the error call back while importing sparkdl ``` ModuleNotFoundError Traceback (most recent call last) in () ----> 1 import sparkdl 1 frames /usr/local/lib/python3.6/dist-packages/sparkdl/image/imageIO.py in () 23 24 #...
Hello I have updated my spark cluster with the new spark release 3.0 But the spark-deep-learning is not working (built with scala v 2.11) Is there any plan to move...
