SynapseML icon indicating copy to clipboard operation
SynapseML copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

HealthCareSDK Returns NullPointerException on Synapse Spark

Open wjohnson opened this issue 2 years ago • 0 comments

Describe the bug When using the HealthCareSDK class in SynapseML, I get a NullPointerException when running on a dataset of 1,000+ rows.

To Reproduce

On a medium amount of data (1,000+ rows) with StringType field between 250 and 4,000 characters long, execute the following code:

%%configure -f
{
  "name": "nerHealthExtract",
  "conf": {
      "spark.jars.packages": "com.microsoft.azure:synapseml_2.12:0.9.5-13-d1b51517-SNAPSHOT",
      "spark.jars.repositories": "https://mmlspark.azureedge.net/maven",
      "spark.jars.excludes": "org.scala-lang:scala-reflect,org.apache.spark:spark-tags_2.12,org.scalactic:scalactic_2.12,org.scalatest:scalatest_2.12",
      "spark.yarn.user.classpath.first": "true"
  }
}

df_text_aggregated = spark.read.parquet("path/to/something")

healthcareService = (HealthcareSDK() 
    .setSubscriptionKey("API_KEY")
    .setLocation("centralus")
    .setErrorCol("nerHealthError")
    .setLanguage("en")
    .setOutputCol("nerHealthOutput"))
df_ner = healthcareService.transform(df_text_aggregated)
df_ner.cache()

df_ner.write.mode("overwrite").parquet("path/to/somewhere/else")

During the write, I receive the StackTrace below.

Expected behavior I would expect to receive the healthcare output across all rows and NOT a NullPointerException.

Info (please complete the following information):

  • SynapseML Version: com.microsoft.azure:synapseml_2.12:0.9.5-13-d1b51517-SNAPSHOT
  • Spark Version 3.1
  • Spark Platform Synapse Spark

Stacktrace

Error: An error occurred while calling o1115.parquet. 
: org.apache.spark.SparkException: Job aborted. 
at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231) 
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:188) 
at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:108) 
at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:106) 
at org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:131) 
at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:218) 
at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:256) 
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) 
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:253) 
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:214) 
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:148) 
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:147) 
at org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:995) 
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:107) 
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:181) 
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:94) 
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) 
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:68) 
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:995) 
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:444) 
at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:416) 
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:294) 
at org.apache.spark.sql.DataFrameWriter.parquet(DataFrameWriter.scala:880) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:498) 
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) 
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
at py4j.Gateway.invoke(Gateway.java:282) 
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) 
at py4j.commands.CallCommand.execute(CallCommand.java:79) 
at py4j.GatewayConnection.run(GatewayConnection.java:238) 
at java.lang.Thread.run(Thread.java:748) 
Caused by : org.apache.spark.SparkException: Job aborted due to stage failure: Task 33 in stage 76.0 failed 4 times, most recent failure: Lost task 33.3 in stage 76.0 (TID 3877) (vm-89521530 executor 1): java.lang.NullPointerException 
at com.azure.ai.textanalytics.implementation.Utility.toRecognizeHealthcareEntitiesResults(Utility.java:510) 
at com.azure.ai.textanalytics.AnalyzeHealthcareEntityAsyncClient.toTextAnalyticsPagedResponse(AnalyzeHealthcareEntityAsyncClient.java:179) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:238) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2194) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2068) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:337) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onNext(MonoCacheTime.java:354) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) 
at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onNext(FluxDoOnEach.java:173) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99) 
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2664) 
at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180) 
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:145) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1817) 
at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:159) 
at reactor.core.publisher.FluxHandle$HandleSubscriber.onComplete(FluxHandle.java:213) 
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete(FluxMap.java:269) 
at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:400) 
at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:419) 
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:473) 
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:684) 
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:93) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) 
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) 
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372) 
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235) 
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284) 
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) 
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) 
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) 
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) 
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) 
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) 
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 
at java.lang.Thread.run(Thread.java:748) Suppressed: java.lang.Exception: #block terminated with an error 
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) 
at reactor.core.publisher.Flux.blockLast(Flux.java:2644) 
at com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:94) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.<init>(ContinuablePagedByItemIterable.java:50) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:37) 
at com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:106) 
at scala.collection.convert.Wrappers$JIterableWrapper.iterator(Wrappers.scala:55) 
at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245) 
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242) 
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108) 
at com.microsoft.azure.synapse.ml.cognitive.HealthcareSDK.invokeTextAnalytics(TextAnalyticsSDK.scala:339) 
at com.microsoft.azure.synapse.ml.cognitive.TextAnalyticsSDKBase.$anonfun$transformTextRows$4(TextAnalyticsSDK.scala:128) 
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) 
at scala.util.Success.$anonfun$map$1(Try.scala:255) 
at scala.util.Success.map(Try.scala:213) 
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) 
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) 
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) 
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) 
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) 
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Driver stacktrace: 
at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2263) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2212) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2211) 
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) 
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) 
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2211) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1082) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1082) 
at scala.Option.foreach(Option.scala:407) 
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1082) 
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2450) 
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2392) 
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2381) 
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49) 
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:869) 
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2282) 
at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:200) ... 33 more 
Caused by : java.lang.NullPointerException 
at com.azure.ai.textanalytics.implementation.Utility.toRecognizeHealthcareEntitiesResults(Utility.java:510) 
at com.azure.ai.textanalytics.AnalyzeHealthcareEntityAsyncClient.toTextAnalyticsPagedResponse(AnalyzeHealthcareEntityAsyncClient.java:179) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:238) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2194) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2068) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:337) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onNext(MonoCacheTime.java:354) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) 
at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onNext(FluxDoOnEach.java:173) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99) 
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2664) 
at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180) 
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:145) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1817) 
at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:159) 
at reactor.core.publisher.FluxHandle$HandleSubscriber.onComplete(FluxHandle.java:213) 
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete(FluxMap.java:269) 
at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:400) 
at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:419) 
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:473) 
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:684) 
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:93) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) 
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) 
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372) 
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235) 
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284) 
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) 
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) 
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) 
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) 
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) 
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) 
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more Suppressed: java.lang.Exception: #block terminated with an error 
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) 
at reactor.core.publisher.Flux.blockLast(Flux.java:2644) 
at com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:94) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.<init>(ContinuablePagedByItemIterable.java:50) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:37) 
at com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:106) 
at scala.collection.convert.Wrappers$JIterableWrapper.iterator(Wrappers.scala:55) 
at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245) 
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242) 
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108) 
at com.microsoft.azure.synapse.ml.cognitive.HealthcareSDK.invokeTextAnalytics(TextAnalyticsSDK.scala:339) 
at com.microsoft.azure.synapse.ml.cognitive.TextAnalyticsSDKBase.$anonfun$transformTextRows$4(TextAnalyticsSDK.scala:128) 
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) 
at scala.util.Success.$anonfun$map$1(Try.scala:255) 
at scala.util.Success.map(Try.scala:213) 
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) 
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) 
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) 
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) 
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) 
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Traceback (most recent call last):   File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 1250, in parquet     self._jwrite.parquet(path)   File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/py4j/java_gateway.py", line 1304, in __call__     return_value = get_return_value(   File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 111, in deco     return f(*a, **kw)   File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/py4j/protocol.py", line 326, in get_return_value     raise Py4JJavaError( py4j.protocol.Py4JJavaError: An error occurred while calling o1115.parquet. : org.apache.spark.SparkException: Job aborted. 
at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:231) 
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.run(InsertIntoHadoopFsRelationCommand.scala:188) 
at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:108) 
at org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:106) 
at org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:131) 
at org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:218) 
at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:256) 
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) 
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:253) 
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:214) 
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:148) 
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:147) 
at org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:995) 
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$5(SQLExecution.scala:107) 
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:181) 
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:94) 
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775) 
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:68) 
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:995) 
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:444) 
at org.apache.spark.sql.DataFrameWriter.saveInternal(DataFrameWriter.scala:416) 
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:294) 
at org.apache.spark.sql.DataFrameWriter.parquet(DataFrameWriter.scala:880) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:498) 
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) 
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
at py4j.Gateway.invoke(Gateway.java:282) 
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) 
at py4j.commands.CallCommand.execute(CallCommand.java:79) 
at py4j.GatewayConnection.run(GatewayConnection.java:238) 
at java.lang.Thread.run(Thread.java:748) 
Caused by : org.apache.spark.SparkException: Job aborted due to stage failure: Task 33 in stage 76.0 failed 4 times, most recent failure: Lost task 33.3 in stage 76.0 (TID 3877) (vm-89521530 executor 1): java.lang.NullPointerException 
at com.azure.ai.textanalytics.implementation.Utility.toRecognizeHealthcareEntitiesResults(Utility.java:510) 
at com.azure.ai.textanalytics.AnalyzeHealthcareEntityAsyncClient.toTextAnalyticsPagedResponse(AnalyzeHealthcareEntityAsyncClient.java:179) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:238) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2194) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2068) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:337) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onNext(MonoCacheTime.java:354) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) 
at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onNext(FluxDoOnEach.java:173) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99) 
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2664) 
at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180) 
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:145) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1817) 
at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:159) 
at reactor.core.publisher.FluxHandle$HandleSubscriber.onComplete(FluxHandle.java:213) 
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete(FluxMap.java:269) 
at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:400) 
at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:419) 
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:473) 
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:684) 
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:93) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) 
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) 
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372) 
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235) 
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284) 
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) 
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) 
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) 
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) 
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) 
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) 
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 
at java.lang.Thread.run(Thread.java:748) Suppressed: java.lang.Exception: #block terminated with an error 
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) 
at reactor.core.publisher.Flux.blockLast(Flux.java:2644) 
at com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:94) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.<init>(ContinuablePagedByItemIterable.java:50) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:37) 
at com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:106) 
at scala.collection.convert.Wrappers$JIterableWrapper.iterator(Wrappers.scala:55) 
at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245) 
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242) 
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108) 
at com.microsoft.azure.synapse.ml.cognitive.HealthcareSDK.invokeTextAnalytics(TextAnalyticsSDK.scala:339) 
at com.microsoft.azure.synapse.ml.cognitive.TextAnalyticsSDKBase.$anonfun$transformTextRows$4(TextAnalyticsSDK.scala:128) 
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) 
at scala.util.Success.$anonfun$map$1(Try.scala:255) 
at scala.util.Success.map(Try.scala:213) 
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) 
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) 
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) 
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) 
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) 
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) Driver stacktrace: 
at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2263) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2212) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2211) 
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) 
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) 
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) 
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2211) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1082) 
at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1082) 
at scala.Option.foreach(Option.scala:407) 
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1082) 
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2450) 
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2392) 
at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2381) 
at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49) 
at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:869) 
at org.apache.spark.SparkContext.runJob(SparkContext.scala:2282) 
at org.apache.spark.sql.execution.datasources.FileFormatWriter$.write(FileFormatWriter.scala:200) ... 33 more 
Caused by : java.lang.NullPointerException 
at com.azure.ai.textanalytics.implementation.Utility.toRecognizeHealthcareEntitiesResults(Utility.java:510) 
at com.azure.ai.textanalytics.AnalyzeHealthcareEntityAsyncClient.toTextAnalyticsPagedResponse(AnalyzeHealthcareEntityAsyncClient.java:179) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:249) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapInner.onSubscribe(MonoFlatMap.java:238) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2194) 
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2068) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:337) 
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onNext(MonoCacheTime.java:354) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2398) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onSubscribe(MonoFlatMap.java:110) 
at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64) 
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) 
at reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143) 
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:157) 
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107) 
at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onNext(FluxDoOnEach.java:173) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:120) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1816) 
at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) 
at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99) 
at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174) 
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) 
at reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2664) 
at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180) 
at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onComplete(FluxDoFinally.java:145) 
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:150) 
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1817) 
at reactor.core.publisher.MonoCollect$CollectSubscriber.onComplete(MonoCollect.java:159) 
at reactor.core.publisher.FluxHandle$HandleSubscriber.onComplete(FluxHandle.java:213) 
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete(FluxMap.java:269) 
at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:400) 
at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:419) 
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:473) 
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:684) 
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:93) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) 
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) 
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372) 
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1235) 
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1284) 
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) 
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) 
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) 
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) 
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) 
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) 
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) 
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more Suppressed: java.lang.Exception: #block terminated with an error 
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99) 
at reactor.core.publisher.Flux.blockLast(Flux.java:2644) 
at com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:94) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.<init>(ContinuablePagedByItemIterable.java:50) 
at com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:37) 
at com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:106) 
at scala.collection.convert.Wrappers$JIterableWrapper.iterator(Wrappers.scala:55) 
at scala.collection.IterableLike.foreach(IterableLike.scala:74) 
at scala.collection.IterableLike.foreach$(IterableLike.scala:73) 
at scala.collection.AbstractIterable.foreach(Iterable.scala:56) 
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245) 
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242) 
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108) 
at com.microsoft.azure.synapse.ml.cognitive.HealthcareSDK.invokeTextAnalytics(TextAnalyticsSDK.scala:339) 
at com.microsoft.azure.synapse.ml.cognitive.TextAnalyticsSDKBase.$anonfun$transformTextRows$4(TextAnalyticsSDK.scala:128) 
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) 
at scala.util.Success.$anonfun$map$1(Try.scala:255) 
at scala.util.Success.map(Try.scala:213) 
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) 
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) 
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) 
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) 
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) 
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) 
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) 
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) 
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

If the bug pertains to a specific feature please tag the appropriate CODEOWNER for better visibility

Additional context Works fine doing a .show() but does not work with the complete dataset.

AB#1817799

wjohnson avatar Jun 03 '22 16:06 wjohnson