deeplake
deeplake copied to clipboard
activeloopai
[FEATURE] Redis storage provider
🚨🚨 Feature Request
- [ ] Related to an existing Issue
- [x] A new implementation (Improvement, Extension)
If your feature will improve HUB
Support Redis as a storage provider for Hub datasets.
Description of the possible solution
Implementations of already supported storage providers can be used as examples: S3 storage, local storage, etc.
Difficulty: Medium
@krishnajalan, apologies for the delay! Yep, similar to how we can store Hub datasets on platforms like S3, we need to implement a way to store Hub datasets as a Redis database.
@up4154 welcome to Hub, and Hacktoberfest! Great to see that you're willing to contribute. Do you have a proposed solution in mind?
Hi @aadityasinha-dotcom! Thanks for your interest in Hub. I just assigned it to you. Have fun!
So we have to implement Redis client just like we implemented the S3 storage.
A small question. Is this the correct implementation for redis client? @farizrahman4u
class ReadisProvider(redis_url):
"""Provider class for using the local filesystem."""
def __init__(self, redis_url):
"""Initializes the RedisProvider.
Args:
redis_url: The url of the provider.
"""
self.r = redis.StrictRedis(url = redis_url, charset = "utf-8", decode_responses = True)
Function to set the value for given bitfield
def set(self, fmt: str, offset: BitfieldOffsetT, value: int):
"""
Set the value of a given bitfield.
:param fmt: format-string for the bitfield being read, e.g. 'u8' for
an unsigned 8-bit integer.
:param offset: offset (in number of bits). If prefixed with a
'#', this is an offset multiplier, e.g. given the arguments
fmt='u8', offset='#2', the offset will be 16.
:param int value: value to set at the given position.
:returns: a :py:class:`BitFieldOperation` instance.
"""
self.operations.append(("SET", fmt, offset, value))
return self
@aadityasinha-dotcom this isnt even valid python: class ReadisProvider(redis_url):, how are you inheriting from a string?
And no, this isnt an implementation of redis provider for hub. You just copied a random snippet from walrus which uses variables (like self.operations) that are defined no where in your code. You are doing the same thing as you did in #1588. This is not acceptable.
Sorry for doing the same thing. I promise I will not do that again. I have referenced this from https://redis.readthedocs.io/en/latest/commands.html#core-commands
From what I found redis is like a dictionary which holds key:value pairs. And I have to implement redis client with its commands like get, set, del, etc.
@farizrahman4u Is it necessary to pass all the parameters?
def __init__(self, host='localhost', port=6379,
db=0, password=None, socket_timeout=None,
socket_connect_timeout=None,
socket_keepalive=None, socket_keepalive_options=None,
connection_pool=None, unix_socket_path=None,
encoding='utf-8', encoding_errors='strict',
charset=None, errors=None,
decode_responses=False, retry_on_timeout=False,
ssl=False, ssl_keyfile=None, ssl_certfile=None,
ssl_cert_reqs='required', ssl_ca_certs=None,
max_connections=None):
