elephas
elephas copied to clipboard
Distributed Deep learning with Keras & Spark
Bumps [pyspark](https://github.com/apache/spark) from 3.2 to 3.2.2. Commits 78a5825 Preparing Spark release v3.2.2-rc1 ba978b3 [SPARK-39099][BUILD] Add dependencies to Dockerfile for building Spark releases 001d8b0 [SPARK-37554][BUILD] Add PyArrow, pandas and plotly to...
It seems to be broken with new versions. I ran [regression](https://github.com/maxpumperla/elephas/blob/master/examples/ml_mlp_regression.py) example on Google colab and gave these results: ``` print(metrics.r2) print(metrics.meanAbsoluteError) print(metrics.rootMeanSquaredError) -57287.893765051085 22.850576681614513 24.626286489524432 ``` _Originally posted by...
Hi everyone, i'have a problem with the method transform. The keras model translate in elephas fit in a pyspark correctly but when i try to transform the validation set, it...
https://github.com/keras-team/keras/blob/master/keras/callbacks.py#L341 The fit function in Keras returns a graph that can be used to determine if the model is overfitting or not. This would be very useful from Elephas.
Hi all Can we write CNN with elephas, i mean if we can add 2d conv layers in keras model and load 2d images. Can we load pretrained models like...
Hi, I'm trying to use elephas for my deep learning models on spark but so far I couldn't even get anything to work on 3 different machines and on multiple...
Dear @maxpumperla / other authors of this repo. A big thanks for developing this library, I have been successful in running it some datasets but I am facing issue with...
Hi, Does this package support LSTM or GRU models for training on spark? Thanks for your great work :)
Hi All I am trying to use a broadcasted dataframe inside a Pyspark UDF and i am getting pickle error when i register the UDF. Please find my code below....