Cannot start Spark session on my laptop

[zhe-thoughts]

I was using very simple code:

import ray
import raydp                                                                
ray.init()
spark = raydp.init_spark(app_name="RayDP example", num_executors=2, executor_cores=2, executor_memory="4GB")

My laptop is MacBook (Big Sur). Tried Ray nightly and 1.2. PySpark 3.0.1. Tried Java 11 and 15.

Got the below error:

Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/jars/ray_dist.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyspark/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" py4j.Py4JNetworkException
	at py4j.GatewayServer.startSocket(GatewayServer.java:788)
	at py4j.GatewayServer.start(GatewayServer.java:763)
	at py4j.GatewayServer.start(GatewayServer.java:746)
	at org.apache.spark.deploy.raydp.AppMasterEntryPoint$.main(AppMasterEntryPoint.scala:39)
	at org.apache.spark.deploy.raydp.AppMasterEntryPoint.main(AppMasterEntryPoint.scala)
Caused by: java.net.BindException: Address already in use
	at java.base/sun.nio.ch.Net.bind0(Native Method)
	at java.base/sun.nio.ch.Net.bind(Net.java:550)
	at java.base/sun.nio.ch.Net.bind(Net.java:539)
	at java.base/sun.nio.ch.NioSocketImpl.bind(NioSocketImpl.java:643)
	at java.base/java.net.ServerSocket.bind(ServerSocket.java:396)
	at py4j.GatewayServer.startSocket(GatewayServer.java:786)
	... 4 more

[ConeyLiu]

Hi Zhe, it seems like the port 25333 is occupied.

发自我的iPhone

在 2021年2月17日，下午2:32，Zhe Zhang [email protected] 写道：



I was using very simple code:

import ray import raydp ray.init() spark = raydp.init_spark(app_name="RayDP example", num_executors=2, executor_cores=2, executor_memory="4GB")

My laptop is MacBook (Big Sur). Tried Ray nightly and 1.2. PySpark 3.0.1. Tried Java 11 and 15.

Got the below error:

Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ray/jars/ray_dist.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyspark/jars/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Exception in thread "main" py4j.Py4JNetworkException at py4j.GatewayServer.startSocket(GatewayServer.java:788) at py4j.GatewayServer.start(GatewayServer.java:763) at py4j.GatewayServer.start(GatewayServer.java:746) at org.apache.spark.deploy.raydp.AppMasterEntryPoint$.main(AppMasterEntryPoint.scala:39) at org.apache.spark.deploy.raydp.AppMasterEntryPoint.main(AppMasterEntryPoint.scala) Caused by: java.net.BindException: Address already in use at java.base/sun.nio.ch.Net.bind0(Native Method) at java.base/sun.nio.ch.Net.bind(Net.java:550) at java.base/sun.nio.ch.Net.bind(Net.java:539) at java.base/sun.nio.ch.NioSocketImpl.bind(NioSocketImpl.java:643) at java.base/java.net.ServerSocket.bind(ServerSocket.java:396) at py4j.GatewayServer.startSocket(GatewayServer.java:786) ... 4 more

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/oap-project/raydp/issues/88, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADBEWSFN7UWYFQ4OLYLYV6TS7NPHHANCNFSM4XXYVQIQ.

[kira-lin]

close as stale