Capture .NET exception in JVM for logging

[AFFogarty]

Output looks like this:

20/10/09 17:04:51 INFO DotnetRunner: Adding key=spark.submit.deployMode and value=client to environment
20/10/09 17:04:51 INFO DotnetRunner: Adding key=spark.master and value=local to environment
Unhandled exception. System.Exception: Test exception
   at SparkDotnet.Program.Main(String[] args) in C:\...\Program.cs:line 16
20/10/09 17:04:54 INFO DotnetRunner: Closing DotnetBackend
Exception in thread "main" org.apache.spark.deploy.dotnet.DotNetUserAppException: User application exited with -532462766 and .NET exception: System.Exception: Test exception
   at SparkDotnet.Program.Main(String[] args) in C:\...\Program.cs:line 16

        at org.apache.spark.deploy.dotnet.DotnetRunner$.main(DotnetRunner.scala:158)
        at org.apache.spark.deploy.dotnet.DotnetRunner.main(DotnetRunner.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
20/10/09 17:04:57 INFO ShutdownHookManager: Shutdown hook called
20/10/09 17:04:57 INFO ShutdownHookManager: Deleting directory C:\...\Temp\spark-7c015b1f-32e5-4581-9189-0d951e11fa85

DotNetUserAppException would get caught by the ApplicationMaster.

---

Without this change, the stdout just looks like this:

20/10/09 17:20:56 INFO DotnetRunner: Adding key=spark.submit.deployMode and value=client to environment
20/10/09 17:20:56 INFO DotnetRunner: Adding key=spark.master and value=local to environment
Unhandled exception. System.Exception: Test exception
   at SparkDotnet.Program.Main(String[] args) in C:\...\Program.cs:line 16
20/10/09 17:20:57 INFO DotnetRunner: Closing DotnetBackend
20/10/09 17:20:57 INFO ShutdownHookManager: Shutdown hook called
20/10/09 17:20:57 INFO ShutdownHookManager: Deleting directory C:\...\Temp\spark-00268866-176f-4051-82fa-c43cb816cbcd

[sunildixit]

@AFFogarty is there a reason this is still in draft? This will be fantastic to get merged in. any idea what's the timeline on this?