SAC : Livy Job submission , Spark Session issue for Atlas . Everything works on Spark-shell and Zepplein Interactive session

[nxverma]

=============================================================================== 20/01/30 18:58:58 INFO YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster registered as NettyRpcEndpointRef(spark://[email protected]:43495) 20/01/30 18:58:58 INFO YarnAllocator: Will request 7 executor container(s), each with 4 core(s) and 7296 MB memory (including 1152 MB of overhead) 20/01/30 18:58:58 INFO YarnAllocator: Submitted 7 unlocalized container requests. 20/01/30 18:58:58 INFO ApplicationMaster: Started progress reporter thread with (heartbeat : 3000, initial allocation : 200) intervals 20/01/30 18:58:58 INFO YarnClusterSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0 20/01/30 18:58:58 INFO YarnClusterScheduler: YarnClusterScheduler.postStartHook done

20/01/30 18:59:21 WARN SparkUtils$: Fail to create Hive Configuration java.lang.IllegalStateException: Cannot find active or default SparkSession in the current context at com.hortonworks.spark.atlas.utils.SparkUtils$.sparkSession(SparkUtils.scala:56) at com.hortonworks.spark.atlas.utils.SparkUtils$.liftedTree1$1(SparkUtils.scala:42) at com.hortonworks.spark.atlas.utils.SparkUtils$.hiveConf(SparkUtils.scala:41) at com.hortonworks.spark.atlas.utils.SparkUtils$$anonfun$2.apply(SparkUtils.scala:68)