DeepImageFeaturizer does not work with Structured Streaming

[mhamilton723]

Somewhere in there you are probably dropping down into RDD apis.

[sueann]

Yeah RDDs are used in TensorFrames which is what is underneath the Featurizer. UDFs can be used as a workaround but we don't have it productionized in this package.

[dimagoldin]

I am getting the following error:

pyspark.sql.utils.AnalysisException: 'Queries with streaming sources must be executed with writeStream.start();;

when doing this:

p_model.transform(imageStream).select("probability", "prediction").writeStream.format("console").start().awaitTermination()

Could you please show an example of the workaround? i wasnt able to use the model correctly as a UDF Thanks!