AttributeError: 'DeepImageFeaturizer' object has no attribute '_to_java'`

[dondrake]

I'm trying to persist my model after I fit it and I get the following exception:

`AttributeError: 'DeepImageFeaturizer' object has no attribute '_to_java'

AttributeError Traceback (most recent call last) in () ----> 1 p_model.write().overwrite().save("dbfs://models/my_model")

/databricks/spark/python/pyspark/ml/pipeline.py in write(self) 210 def write(self): 211 """Returns an MLWriter instance for this ML instance.""" --> 212 return JavaMLWriter(self) 213 214 @since("2.0.0")

/databricks/spark/python/pyspark/ml/util.py in init(self, instance) 98 def init(self, instance): 99 super(JavaMLWriter, self).init() --> 100 _java_obj = instance._to_java() 101 self._jwrite = _java_obj.write() 102

/databricks/spark/python/pyspark/ml/pipeline.py in _to_java(self) 247 java_stages = gateway.new_array(cls, len(self.stages)) 248 for idx, stage in enumerate(self.stages): --> 249 java_stages[idx] = stage._to_java() 250 251 _java_obj =\

AttributeError: 'DeepImageFeaturizer' object has no attribute '_to_java'`

Is there a workaround for this?

Thanks.

[sueann]

@jkbradley @ajaysaini725 - would this be fixed / fixable with the python custom transformer persistence work?

[VedAustin]

@sueann, @jkbradley, @phi-dbq, @ajaysaini725 : Any updates on this? Or Examples/workarounds how custom transformers could be persisted. I am on Azure Databricks (spark version: 2.2.1)

[sueann]

@VedAustin this has been fixed in master. Release 0.3.0 provides persistence for DeepImageFeaturizer in Scala (PR) but not in Python. Release 1.0.0 which should be out in a few weeks provides persistence for DeepImageFeaturizer in Scala and Python (PR), but will no longer support older versions of Spark (only 2.3 and higher).

[VedAustin]

@sueann Thanks for coming back. Sorry, I misunderstood: I was under the impression that DeepImageFeaturizer was a custom transformer. My problem is more related to saving to disk, a more custom created transformer, like this: https://stackoverflow.com/questions/49311339/save-custom-transformers-in-pyspark and is related to this: https://issues.apache.org/jira/browse/SPARK-17025

If this fixed, is it only available in >=2.3?