SynapseML
SynapseML copied to clipboard
microsoft
Simple and Distributed Machine Learning
Ports two of the basic python notebook examples to scala using zeppelin notebooks for https://github.com/Azure/mmlspark/issues/166 and https://github.com/Azure/mmlspark/issues/453 Github doesn't have a native viewer for zeppelin, so I included instructions on...
The _activateCondaEnv_ command works on Windows but returns an empty Seq() for other OS's. As a consequence when running, for example, _installPipPackage_ on Linux the Python wheels are installed under...
**Describe the bug** ```LightGBMClassificationModel.fit``` Cannot handle too much data. Fails without even having to collect anything at the driver. I ```LightGBMClassificationModel.fit``` on data(10000,241) - It executes perfectly I ```LightGBMClassificationModel.fit``` on...
First, I tried **2 spark slaves**, it take about 11 minutes to train my model. submit info: spark-submit --master yarn **--num-executors 2** --executor-memory 19G --executor-cores 16 --conf spark.dynamicAllocation.enabled=false --jars s3://EMR/jars/synapseml-vw_2.12-0.9.4.jar,s3://EMR/jars/synapseml_2.12-0.9.4.jar,s3://EMR/jars/client-sdk-1.14.0.jar...
I am trying to use hyperopt to tune the hyperparameters for the LightGBM model and sometime I get this error `Py4JJavaError('An error occurred while calling o527.fit.\n', JavaObject id=o680)`. In my...
With pyspark 3.2.0 now supporting ```pandas_on_pyspark``` (earlier koalas), this would make things much easier for a lot of us travelling from the faraway world of pandas. Currently, we need to...
**Describe the bug** is there anyone tried to build mmlspark on your own desktop ? some unit tests could not pass for the LightGBM modules, such as com.microsoft.ml.spark.lightgbm.split1.VerifyLightGBMClassifier#"Verify LightGBM Classifier...
When installing spark via miniconda the spark home is not set so you cannot run spark in AML. You will get errors when you try to set the spark context...
