Add ability to stop submitted Spark applications in SDK

Open jafreck opened this issue 7 years ago • 2 comments

In interactive mode you can kill running applications through the master ui, but there is no equivalent in the SDK.

This would allow application timeout/retries.

Apr 02 '18 21:04 jafreck

AZTK spark submit extra parameters with maximum retry, max time to run for application

Apr 02 '18 22:04 praneet22

Looking for some feature to perform task using AZTK SDK mentioned in spark Doc here

Additionally, standalone cluster mode supports restarting your application automatically if it exited with non-zero exit code. To use this feature, you may pass in the --supervise flag to spark-submit when launching your application. Then, if you wish to kill an application that is failing repeatedly, you may do so through:

./bin/spark-class org.apache.spark.deploy.Client kill <driver ID>

Apr 05 '18 18:04 praneet22