shc
shc copied to clipboard
Set hbase host & port with code
Is it possible to provide spark.hbase.host and spark.hbase.port in code? I tried following
`val sparkConf = new SparkConf().setAppName("Spark-HBase").setMaster("local[4]")
sparkConf.set("spark.hbase.host", "hbase_dev.myorg.net");
val spark: SparkSession = SparkSession.builder()
.config(sparkConf)
.getOrCreate()`
but getting error as shc tries to connect to localhost:
19/09/25 21:16:34 INFO ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) 19/09/25 21:16:34 WARN ReadOnlyZKClient: 0x1a785fd5 to localhost:2181 failed for get of /hbase/hbaseid, code = CONNECTIONLOSS, retries = 3 19/09/25 21:16:34 WARN ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
I know it is a long shot, but did you ever solve it, and how?