luphappy

@lw-lin 您好，请教个问题，我现在从flume去数据，运行spark streaming的程序，设置了spark.streaming.blockInterval为1000ms，但感觉没有起作用，这种情况遇到过么，谢谢

我用spark streaming，但时间长了之后，经常会自己就退出了，是为什么呢

您好，请问，我的spark-streaming程序以yarn-client的方式运行了一段时间后，就退出了，但driver还在，yarn日志如下： 15/09/19 14:32:34 ERROR util.Utils: Uncaught exception in thread Thread-1 org.apache.hadoop.security.token.SecretManager$InvalidToken: Invalid AMRMToken from appattempt_1437371132890_10529_000002 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53) at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.finishApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:94)...

@lw-lin 2.6.0，应该是那个原因

您好，我想问一下，如果要是想catch spark Exception，应该怎么catch JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, batchInterval); ``` JavaPairInputDStream logs = KafkaUtils .createDirectStream(jssc, String.class, String.class, StringDecoder.class, StringDecoder.class, kafkaParams, topicsSet); dotask() ``` 这个应该怎么catch才能让application不退出呢

您好，我的spark streaming程序是以yarn-client方式运行的，但每运行一段时间，就会自己退出了，日志如下： ERROR util.Utils: Uncaught exception in thread Thread-1 org.apache.hadoop.security.token.SecretManager$InvalidToken: Invalid AMRMToken from appattempt_1451463897786_21220_000002 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53) at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104) at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.finishApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:94) at...