Laneser

Results 3 comments of Laneser

Thanks for your information! I will try to setup the dotnet spark jvm bridge through the `spark.driver.extraClassPath` and let `buildSparkSubmitCommand` do the same things for dotnet spark.

I think the JVMBridgeHelper works like pyspark launch gateway method https://github.com/apache/spark/blob/feeca63198466640ac461a2a34922493fa6162a8/python/pyspark/context.py#L319 https://github.com/apache/spark/blob/95aec091e4d8a45e648ce84d32d912f585eeb151/python/pyspark/java_gateway.py#L40 The SparkConf could not work if jvm bridge does not exists. I will try to solve the E2E...

Though the checks have passed, but I have no idea why it was not working before... 😿