Answer from Java side is empty - MMLspark

[naadvar]

Hi all,

I am running the following config on Databricks Scala: 2.12 Spark: 3.1.2

I installed the following jars for mmlspark in Databricks:

• mmlspark-1.0.0-rc4.jar
• mmlspark-core-1.0.0-rc4.jar
• mmlspark-lightgbm-1.0.0-rc4.jar

When I run the following lines, I get a 'Answer from Java side is empty' error:

from mmlspark.lightgbm import LightGBMClassifier

classifier = LightGBMClassifier( featuresCol="features", labelCol="ARR_DEL15", learningRate=0.3, numIterations=150, numLeaves=100)

Are the Jars I have the right ones for spark 3 + ? or is there any other solution to this?

AB#1884836

[imatiach-msft]

@naadvar sorry about the trouble you are having. Can you try the latest version of synapseml instead of mmlspark rc4? mmlspark was renamed to synapseml. Do you still see the same issue?

[ppruthi]

@naadvar -- can you confirm if you still the issue post the use of synapseml instead of mmlspark ?

[ss-github-code]

I also get the same error on a dual CPU Linux Ubuntu pc (not Databricks):

MMLSpark version: com.microsoft.azure:synapseml_2.12:0.9.5 System version: 3.9.12 (main, Apr 5 2022, 06:56:58) [GCC 7.5.0] PySpark version: 3.3.0

running LightGBMClassifier on the full Criteo train.txt using the recommenders/examples/02_model_content_based_filtering/mmlspark_lightgbm_criteo.ipynb;

model = lgbm.fit(train)

---

ERROR:root:Exception while sending command.====================> (19 + 1) / 20] Traceback (most recent call last): File "/home/shiv/anaconda3/envs/RecSys_39/lib/python3.9/site-packages/py4j/clientserver.py", line 516, in send_command raise Py4JNetworkError("Answer from Java side is empty") py4j.protocol.Py4JNetworkError: Answer from Java side is empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/shiv/anaconda3/envs/RecSys_39/lib/python3.9/site-packages/py4j/java_gateway.py", line 1038, in send_command response = connection.send_command(command) File "/home/shiv/anaconda3/envs/RecSys_39/lib/python3.9/site-packages/py4j/clientserver.py", line 539, in send_command raise Py4JNetworkError( py4j.protocol.Py4JNetworkError: Error while sending or receiving