Oleksii Diagiliev
Oleksii Diagiliev
@iyer-r , Do you Scala or Python? How do you convert Byte Array to String? Please share you code. If you use Python, I guess it might be because bytes...
Hi @raj-blis , I didn't understand what you try to do. Your input dataframe has a string `ABCD22A54B2E4368A0653A47897E50B0`, why do you convert it to byte array (first snippet) or another...
I ran with a `raw` option: `redis-cli --raw` and see that the string is the same as in spark before saving: ``` 127.0.0.1:6379> keys * test:«Í"¥K.Ch e:G~P° _spark:test:schema ``` Also,...
Hi @raj-blis , Thanks for the detailed explanation. You are right, `spark-redis` uses string based API. I will take a look if we store raw byte array without converting to...
Hi @raj-blis , in [HashRedisPersistence](https://github.com/RedisLabs/spark-redis/blob/6aaaba200aaacc02479a77a3b7fe3fa7ba74c2ec/src/main/scala/org/apache/spark/sql/redis/HashRedisPersistence.scala#L20) we use string oriented Redis API to write and read data, i.e. `Pipeline.hmSet()` takes string parameters. This API works with `byte[]` alternatively and I guess...
@raj-blis , no, my suggestion was to implement `RDD` support for byte arrays, not DataFrame. It will be much easier. The API will be similar to this [one](https://github.com/RedisLabs/spark-redis/blob/master/doc/rdd.md#strings-1), but it...
@raj-blis Okay, we can implement a function `spark.toRedisList()` that takes `RDD[(Array[Byte], Seq[Array[Byte]])]` as an argument. It will store data in [Redis List](https://redis.io/topics/data-types#lists). RDD type is a tuple where first element...
Hi @raj-blis , I have implemented the function in this branch https://github.com/RedisLabs/spark-redis/tree/issue-205-toRedisByteLIST You should find the usage example in the unit test. Could you please test it with your use...
HI @yougaindra There is no such API, but you can implement it yourself, here is a similar example https://github.com/RedisLabs/spark-redis/issues/214
Hi @yougaindra , the approach described in #214 should be more efficient. The approach you mentioned will load all the keys on the driver and then trigger a separate spark...