shc icon indicating copy to clipboard operation
shc copied to clipboard

Set hbase host & port with code

Open mohitgargk opened this issue 6 years ago • 1 comments

Is it possible to provide spark.hbase.host and spark.hbase.port in code? I tried following

`val sparkConf = new SparkConf().setAppName("Spark-HBase").setMaster("local[4]")

sparkConf.set("spark.hbase.host", "hbase_dev.myorg.net");

val spark: SparkSession = SparkSession.builder()
  .config(sparkConf)
  .getOrCreate()`

but getting error as shc tries to connect to localhost:

19/09/25 21:16:34 INFO ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) 19/09/25 21:16:34 WARN ReadOnlyZKClient: 0x1a785fd5 to localhost:2181 failed for get of /hbase/hbaseid, code = CONNECTIONLOSS, retries = 3 19/09/25 21:16:34 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)

mohitgargk avatar Sep 25 '19 15:09 mohitgargk

I know it is a long shot, but did you ever solve it, and how?

injulkarnilesh avatar Oct 29 '20 14:10 injulkarnilesh