Error Training LGBM Classifier on Databricks

[liuyu930]

HI, When I tried to train a classification model using LightGBM in a Databricks Serverless environment, I encountered the following error: ['str' object has no attribute 'getParam']

I cannot confirm whether it's an environment issue or a package issue.

However, I am currently attempting to pinpoint the potential environment-related problems.

If anyone has any insights, I would greatly appreciate your guidance.

environment Databricks Serverless 4 pyspark : 4.0.0

code

`model = LightGBMClassifier( objective="binary", featuresCol="features", labelCol="buy_flg", isUnbalance=True )

model = model.fit(trin_date)`

Error

AttributeError: 'str' object has no attribute 'getParam' [Trace ID: 00-678e75eae65d2f9e8bc744eb078227e0-eb6e67874598c4bf-00] File <command-4855832797700963>, line 33 27 # trin_date.display() 29 model = LightGBMClassifier( 30 objective="binary", featuresCol="features", labelCol="buy_flg", isUnbalance=True 31 ) ---> 33 model = model.fit(trin_date) File <command-4855832797700963>, line 33 27 # trin_date.display() 29 model = LightGBMClassifier( 30 objective="binary", featuresCol="features", labelCol="buy_flg", isUnbalance=True 31 ) ---> 33 model = model.fit(trin_date) File /databricks/python_shell/lib/dbruntime/MLWorkloadsInstrumentation/_pyspark.py:30, in _create_patch_function.<locals>.patched_method(self, *args, **kwargs) 28 call_succeeded = False 29 try: ---> 30 result = original_method(self, *args, **kwargs) 31 call_succeeded = True 32 return result File /databricks/python/lib/python3.12/site-packages/pyspark/ml/base.py:203, in Estimator.fit(self, dataset, params) 201 return self.copy(params)._fit(dataset) 202 else: --> 203 return self._fit(dataset) 204 else: 205 raise TypeError( 206 "Params must be either a param map or a list/tuple of param maps, " 207 "but got %s." % type(params) 208 ) File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b17e7c41-2947-4628-8639-a4f59e8d5edb/lib/python3.12/site-packages/synapse/ml/lightgbm/LightGBMClassifier.py:2148, in LightGBMClassifier._fit(self, dataset) 2147 def _fit(self, dataset): -> 2148 java_model = self._fit_java(dataset) 2149 return self._create_model(java_model) File /databricks/python/lib/python3.12/site-packages/pyspark/ml/wrapper.py:412, in JavaEstimator._fit_java(self, dataset) 397 """ 398 Fits a Java model to the input dataset. 399 (...) 408 fitted Java model 409 """ 410 assert self._java_obj is not None --> 412 self._transfer_params_to_java() 413 return self._java_obj.fit(dataset._jdf) File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b17e7c41-2947-4628-8639-a4f59e8d5edb/lib/python3.12/site-packages/synapse/ml/core/schema/Utils.py:134, in ComplexParamsMixin._transfer_params_to_java(self) 132 self._java_obj.set(pair) 133 if self.hasDefault(param): --> 134 pair = self._make_java_param_pair(param, self._defaultParamMap[param]) 135 pair_defaults.append(pair) 136 if len(pair_defaults) > 0: File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b17e7c41-2947-4628-8639-a4f59e8d5edb/lib/python3.12/site-packages/synapse/ml/core/serialize/java_params_patch.py:122, in _mml_make_java_param_pair(self, param, value) 120 sc = SparkContext._active_spark_context 121 param = self._resolveParam(param) --> 122 java_param = self._java_obj.getParam(param.name) 123 java_value = _mml_py2java(sc, value) 124 return java_param.w(java_value)