sparkflow icon indicating copy to clipboard operation
sparkflow copied to clipboard

Published 20 hours ago • verified publisher icon lifeomic

Keras support?

Open nirmalsinghania2008 opened this issue 5 years ago • 5 comments

Hi Devs, I really liked the clean API of sparkflow for distributed training. Is it possible to run keras code using sparkflow?

nirmalsinghania2008 avatar Jan 28 '19 14:01 nirmalsinghania2008

I haven't actually tried this, but I am guessing you could use tf.keras.layers (and maybe the default keras layers? Not sure). The optimizers will still need to be tensorflow though. We provided an abstraction for some of the optimizers in sparkflow.graph_utils, and just like keras, you can specify by string the optimizer that you want to use.

dmmiller612 avatar Jan 28 '19 14:01 dmmiller612

tf.keras should work I guess, will check.

Thanks.

nirmalsinghania2008 avatar Jan 29 '19 06:01 nirmalsinghania2008

Hi, are there any updates on this thread? Has tf.keras been tried and does this work?

Thanks

Marijse-vdberg avatar Apr 11 '19 09:04 Marijse-vdberg

When I get some time, I can provide an example. Since the graph gets created the same tf.leras should definitely work. You just won't be able to use the model.compile, etc from keras. I am currently working on some updates for tensorflow 2.0, where keras is a major feature in that release.

dmmiller612 avatar Apr 12 '19 14:04 dmmiller612

I think you are on the right track by focusing on TF 2.0 where Keras is a first-class citizen. Also, TF 2.0 is much cleaner and much easier to use, so thanks for working on it @dmmiller612 👍

maziyarpanahi avatar Jun 26 '19 17:06 maziyarpanahi