hbrdd

hbrdd copied to clipboard

开启checkpoint后异常

2

18/08/22 15:50:41 ERROR Utils: Exception encountered java.io.NotSerializableException: org.apache.hadoop.conf.Configuration at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) 我这是spark streaming任务 stream.foreach{rdd=>rdd.put2hase(....)} 应该是put2hbase方法中：def put2Hbase(tableName: String)(implicit config: HbRddConfig) 调用了config.getHbaseConfig 然后这个方法实现是：def getHbaseConfig = HBaseConfiguration.create(config) create方法中创建了Configuration conf = new Configuration();　　...

hehuiyuan