spark icon indicating copy to clipboard operation
spark copied to clipboard

Install Spark Exception

Open gnedash opened this issue 3 years ago • 2 comments

Problem encountered on https://dotnet.microsoft.com/learn/data/spark-tutorial/install-spark Operating System: windows I couldn't setup Spark. I got an error after command spark-submit --version

Exception in thread "main" java.lang.ExceptionInInitializerError at org.apache.spark.unsafe.array.ByteArrayMethods.(ByteArrayMethods.java:54) at org.apache.spark.internal.config.package$.(package.scala:1006) at org.apache.spark.internal.config.package$.(package.scala) at org.apache.spark.deploy.SparkSubmitArguments.$anonfun$loadEnvironmentArguments$3(SparkSubmitArguments.scala:157) at scala.Option.orElse(Option.scala:447) at org.apache.spark.deploy.SparkSubmitArguments.loadEnvironmentArguments(SparkSubmitArguments.scala:157) at org.apache.spark.deploy.SparkSubmitArguments.(SparkSubmitArguments.scala:115) at org.apache.spark.deploy.SparkSubmit$$anon$2$$anon$3.(SparkSubmit.scala:990) at org.apache.spark.deploy.SparkSubmit$$anon$2.parseArguments(SparkSubmit.scala:990) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:85) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make private java.nio.DirectByteBuffer(long,int) accessible: module java.base does not "opens java.nio" to unnamed module @5e5d171f at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:357) at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297) at java.base/java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:188) at java.base/java.lang.reflect.Constructor.setAccessible(Constructor.java:181) at org.apache.spark.unsafe.Platform.(Platform.java:56)

I've used this tutorial https://dotnet.microsoft.com/learn/data/spark-tutorial/install-spark My OS Windows 10 JAVA_HOME path C:\Program Files\Java\jdk-16

gnedash avatar Apr 09 '21 17:04 gnedash

@gnedash can you install java 8 and try again? https://dotnet.microsoft.com/learn/data/spark-tutorial/install-pre-reqs

suhsteve avatar Apr 09 '21 22:04 suhsteve

I can't even get this far along the tutorial, you must be lucky...

image

image

Devtr0n avatar Sep 23 '22 04:09 Devtr0n