Alex Liu comments

Repositories
Issues
Comments

Results 2 comments of


                                            Alex Liu

[BUG]: Error while running a .net application containing a UDF against remote databricks apache spark (e.g. Azure Databricks)

Another way around is using the compatible python version in dotNet app so it matches spark cluster. How does driver write UDF and send them to spark cluster? It could...

[BUG]: Error while running a .net application containing a UDF against remote databricks apache spark (e.g. Azure Databricks)

Are you sure that spark cluster runs Microsoft.Spark.Worker.exe? Why Microsoft.Spark.Worker.exe uses python version "0.4.0" which is quite old.