Error while trying to use use AAD Authentication to Connect Event Hubs with secrets in callback

[Silverutm]

Bug Report:

• Actual behavior
  • Error message

Job aborted due to stage failure: Task 83 in stage 293.0 failed 4 times, most recent failure: Lost task 83.3 in stage 293.0 (TID 33592, XXXXX.cloudapp.net, executor 2): java.util.concurrent.CompletionException: java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1606)
	at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Caused by: java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
	at com.microsoft.aad.msal4j.TokenRequestExecutor.createOauthHttpRequest(TokenRequestExecutor.java:40)
	at com.microsoft.aad.msal4j.TokenRequestExecutor.executeTokenRequest(TokenRequestExecutor.java:31)
	at com.microsoft.aad.msal4j.AbstractClientApplicationBase.acquireTokenCommon(AbstractClientApplicationBase.java:128)
	at com.microsoft.aad.msal4j.AcquireTokenByAuthorizationGrantSupplier.execute(AcquireTokenByAuthorizationGrantSupplier.java:63)
	at com.microsoft.aad.msal4j.AcquireTokenByClientCredentialSupplier.acquireTokenByClientCredential(AcquireTokenByClientCredentialSupplier.java:86)
	at com.microsoft.aad.msal4j.AcquireTokenByClientCredentialSupplier.execute(AcquireTokenByClientCredentialSupplier.java:49)
	at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:71)
	at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:20)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	... 5 more
Caused by: java.lang.ClassNotFoundException: javax.mail.internet.ParseException
	at java.lang.ClassLoader.findClass(ClassLoader.java:524)
	at org.apache.spark.util.ParentClassLoader.findClass(ParentClassLoader.java:35)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
	at org.apache.spark.util.ParentClassLoader.loadClass(ParentClassLoader.java:40)
	at org.apache.spark.util.ChildFirstURLClassLoader.loadClass(ChildFirstURLClassLoader.java:48)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
	... 14 more

• Expected behavior
  • Able to write to eventhubs
• Spark version
  • sparkVersion = "2.4.4"
• spark-eventhubs artifactId and version
  • "com.microsoft.azure" %% "azure-eventhubs-spark" % "2.3.21"

I'm using the code mentioned in https://github.com/Azure/azure-event-hubs-spark/blob/master/docs/use-aad-authentication-to-connect-eventhubs.md with Write Secrets in Callback Class

Codes looks like

[LHuang2019]

@Silverutm did you happen to find the solution for this?

[idkburkes]

I am having this same issue. @Silverutm Any luck here?