mleap icon indicating copy to clipboard operation
mleap copied to clipboard

Add support for spark-deep-learning (e.g. DeepImageFeaturizer)

Open FredYao opened this issue 5 years ago • 4 comments

Databricks has its own packge for running Deep Learning on Spark: https://github.com/databricks/spark-deep-learning Wondering if MLeap supports this or not? Thanks!

FredYao avatar Jul 17 '19 15:07 FredYao

Hey @FredYao, no there's no direct support for the deep learning transformers under spark-deep-learning, but mleap does support tensorflow/keras if that is what you're looking after.

I'm happy to review a PR if you'd like to add them in, thank you!

ancasarb avatar Jul 19 '19 13:07 ancasarb

@ancasarb , thank you for your response. I was checking the mleap document (http://mleap-docs.combust.ml/) for TensorFlow support, however, I barely can find any examples or guidances for using that. Is there any code example that I can refer to? Thanks!

FredYao avatar Jul 19 '19 15:07 FredYao

There are some TF examples in our unit tests https://github.com/combust/mleap/tree/master/mleap-tensorflow/src/test/scala/ml/combust/mleap/tensorflow. Please let me know if you have any questions.

ancasarb avatar Sep 24 '19 08:09 ancasarb

@ancasarb I don't see any examples of taking a Tensorflow graph created in Tensorflow/Keras and serializing it into MLeap.

The example you linked is Scala code that uses MLeaps custom built TensorflowModel object.

Does MLeap support taking a true Tensorflow/Keras model and serializing it? If so could you please provide an example of doing so?

Thank you!

Ben-Epstein avatar Mar 16 '20 21:03 Ben-Epstein