spark-state-store

spark-state-store copied to clipboard

Does it support HDFS as state store ?

1

would like to know, does it support hdfs as state store ?

itsmesrds

No SST files found

5

I'm trying to use this with structure streaming jobs and all my queries terminate with [this](https://github.com/qubole/spark-state-store/blob/bc47e67afdac68bbed0daeefdf6686cd4d5396af/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDbStateStoreProvider.scala#L435) exception > Error reading snapshot file $fileToRead of $this: No SST files found. Do...

aandis

Unable to perform stream-stream Join using Spark-shell

3

Hi, I am having `java.lang.IllegalStateException: Error while creating OptimisticTransactionDb instance lock : file:/home/centos/rocksdb/rdb/db/state_-1619083917/0/0/LOCK: No locks available. `Error while running stream-stream Join on spark-shell. **Step to reproduce.** 1. Spark version 2.4.0...

nbalusu