training icon indicating copy to clipboard operation
training copied to clipboard

Can't restart spark-shell after getting disconnected from master

Open mhlr opened this issue 11 years ago • 0 comments

I got disconnected from my aws master. After I logging back in I can not run spark-shell anymore. Here is what I am getting:

[root@ip-10-170-21-110 ~]# /root/spark/spark-shell Welcome to


/ / ___ / / \ / _ / _ `/ __/ '/ // .__/,// //_\ version 0.8.0 /_/

Using Scala version 2.9.3 (OpenJDK 64-Bit Server VM, Java 1.7.0_25) Initializing interpreter... 13/08/29 20:47:17 INFO server.Server: jetty-7.6.8.v20121106 13/08/29 20:47:17 INFO server.AbstractConnector: Started [email protected]:41680 Creating SparkContext... 13/08/29 20:47:24 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started 13/08/29 20:47:24 INFO spark.SparkEnv: Registering BlockManagerMaster 13/08/29 20:47:25 INFO storage.MemoryStore: MemoryStore started with capacity 8.0 GB. 13/08/29 20:47:25 INFO storage.DiskStore: Created local directory at /mnt/spark/spark-local-20130829204725-4136 13/08/29 20:47:25 INFO storage.DiskStore: Created local directory at /mnt2/spark/spark-local-20130829204725-c34b 13/08/29 20:47:25 INFO storage.DiskStore: Created local directory at /mnt3/spark/spark-local-20130829204725-7948 13/08/29 20:47:25 INFO storage.DiskStore: Created local directory at /mnt4/spark/spark-local-20130829204725-60e8 13/08/29 20:47:25 INFO network.ConnectionManager: Bound socket to port 48385 with id = ConnectionManagerId(ip-10-170-21-110.ec2.internal,48385) 13/08/29 20:47:25 INFO storage.BlockManagerMaster: Trying to register BlockManager 13/08/29 20:47:25 INFO storage.BlockManagerMaster: Registered BlockManager 13/08/29 20:47:25 INFO server.Server: jetty-7.6.8.v20121106 13/08/29 20:47:25 INFO server.AbstractConnector: Started [email protected]:52193 13/08/29 20:47:25 INFO broadcast.HttpBroadcast: Broadcast server started at http://10.170.21.110:52193 13/08/29 20:47:25 INFO spark.SparkEnv: Registering MapOutputTracker 13/08/29 20:47:25 INFO spark.HttpFileServer: HTTP File server directory is /tmp/spark-ff265713-8279-4690-ae69-74077a9481b6 13/08/29 20:47:25 INFO server.Server: jetty-7.6.8.v20121106 13/08/29 20:47:25 INFO server.AbstractConnector: Started [email protected]:58875 13/08/29 20:47:25 INFO server.Server: jetty-7.6.8.v20121106 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/storage/rdd,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/storage,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/stages/stage,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/stages/pool,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/stages,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/environment,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/executors,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/metrics/json,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/static,null} 13/08/29 20:47:25 INFO handler.ContextHandler: started o.e.j.s.h.ContextHandler{/,null} 13/08/29 20:47:25 INFO server.AbstractConnector: Started [email protected]:3030 13/08/29 20:47:25 INFO ui.SparkUI: Started Spark Web UI at http://ip-10-170-21-110.ec2.internal:3030 13/08/29 20:47:25 INFO client.Client$ClientActor: Connecting to master spark://ec2-54-234-132-234.compute-1.amazonaws.com:7077 Spark context available as sc. 13/08/29 20:47:25 ERROR client.Client$ClientActor: Connection to master failed; stopping client 13/08/29 20:47:25 ERROR cluster.SparkDeploySchedulerBackend: Disconnected from Spark cluster! 13/08/29 20:47:25 ERROR cluster.ClusterScheduler: Exiting due to error from cluster scheduler: Disconnected from Spark cluster 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/static,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/metrics/json,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/executors,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/environment,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/stages,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/stages/pool,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/stages/stage,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/storage,null} 13/08/29 20:47:25 INFO handler.ContextHandler: stopped o.e.j.s.h.ContextHandler{/storage/rdd,null} [root@ip-10-170-21-110 ~]#

thanks Daniel

mhlr avatar Aug 29 '13 21:08 mhlr