metorikku icon indicating copy to clipboard operation
metorikku copied to clipboard

Published 20 hours ago • verified publisher icon YotpoLtd

Need to add dependencies in built.sbt

Open sourav2050 opened this issue 3 years ago • 3 comments

Hi, You need to add below two dependencies in built.sbt file at below your ibraryDependencies. I am using Spark version 2.4.4 and Scala version 2.11.12.

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.4.4" libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.4"

As without these two dependencies the code is throwing error NoClassDefFoundError for SparkSession, Spark Listener and logger log4j.

sourav2050 avatar Jul 25 '20 19:07 sourav2050

What IDE Are you using? (as these are included as provided)

lyogev avatar Jul 26 '20 07:07 lyogev

I am running my code in Intellij. Although those dependencies are added in your built.sbt but still that throwing error with class not found. I added above two dependencies below your library dependencies and code ran fine. Does it matter on the basis of IDE?

sourav2050 avatar Jul 26 '20 14:07 sourav2050

image Use this feature in intellij to get the provided scoped dependencies

lyogev avatar Jul 26 '20 14:07 lyogev