sparkflow
sparkflow copied to clipboard
lifeomic
Easy to use library to bring Tensorflow on Apache Spark
We're wanting to use SparkFlow for source detection on radio astronomy images, but even on the MNINST example jobs with larger iteration counts are failing. Given the 60,000 images in...
I cloned sparkflow and run with my TF1.15. The runtime log multiple errors which relate to sparkflow So i make some changes toward sparkflow on master branch and all the...
I'm running the below example in tf 2.0 and I get an error `AttributeError: module 'tensorflow_core._api.v2.train' has no attribute 'export_meta_graph'`. It seems the build_graph function is incompatible with the tf...
When initializing SparkAsyncDL with optimizer='adam', we get the following exception Traceback (most recent call last): File "spark-train.py", line 170, in p = Pipeline(stages=[va, spark_model]).fit(train) File "/usr/local/spark/python/pyspark/ml/base.py", line 132, in fit...
@dmmiller612 Sparkflow is a great API ! Thanks for the super work. Are there any plans to developing a TimeSeriesGenerator wrapper similar to this one: [Tensorflow TimeSeriesGenerator](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/sequence/TimeseriesGenerator) I have a...
Hello, thank you very much for this beautiful project. I try to test mnist example on our cluster but so far without any success :( The first iteration of training...
Hey guys, just a minor thing. Since I run most of my analysis using a Zeppelin frontend, I also wanted to use it when training models with SparkFlow. Sadly tho,...
With spark 2.4.0, Barrier Executors were added to ensure tasks run at the same time. We should add this for training in SparkFlow.
Hi Devs, I really liked the clean API of sparkflow for distributed training. Is it possible to run keras code using sparkflow?
