feat: floyd supports one key for one data structure
背景
当前 Pika 的一个 Key 可以对应多种数据结构,和 Redis 不一致
解决方案
在 Floyd 现有的设计之上,将之前 String 类型所在的Column-Family 用于存放所有的 Key 的 Meta 信息
修改前:
enum ColumnFamilyIndex {
kStringsCF = 0,
kHashesMetaCF = 1,
kHashesDataCF = 2,
kSetsMetaCF = 3,
kSetsDataCF = 4,
kListsMetaCF = 5,
kListsDataCF = 6,
kZsetsMetaCF = 7,
kZsetsDataCF = 8,
kZsetsScoreCF = 9,
kStreamsMetaCF = 10,
kStreamsDataCF = 11,
};
修改后:
enum ColumnFamilyIndex {
kMetaCF = 0,
kHashesDataCF = 1,
kSetsDataCF = 2,
kListsDataCF = 3,
kZsetsDataCF = 4,
kZsetsScoreCF = 5,
kStreamsDataCF = 6,
};
kMetaCF 字段设计
修改前
String
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
| value | reserve | cdate | timestamp |
| | 16B | 8B | 8B |
Hash
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
| hash_size | version | reserve | cdate | timestamp |
| 4B | 8B | 16B | 8B | 8B |
List
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
|list_size| version | left index | right index | reserve | cdate | timestamp |
| 8B | 8B | 8B | 8B | 16B | 8B | 8B |
set
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
| set_size | version | reserve | cdate | timestamp |
| 4B | 8B | 16B | 8B | 8B |
zset
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
| zset_size | version | reserve | cdate | timestamp |
| 4B | 8B | 16B | 8B | 8B |
Stream
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
| group_id_ | entries_added_ | first_id_ms | first_id_seq | last_id_ms | last_id_seq | max_deleted_entry_ms | max_deleted_entry_seq | length | version |
| 4B | 8B | 8B | 8B | 8B | 8B | 8B | 8B | 4B | 4B |
修改后
我们对每一种数据类型的 Meta 的 value 前增加一个字段 Type 用于区别每个 Key 对应的数据结构
以 Set 类型举例
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
| type | set_size | version | reserve | cdate | timestamp |
| 1B | 4B | 8B | 16B | 8B | 8B |
String 的 Meta 格式
key格式
| reserve1 | key | reserve2 |
| 8B | | 16B |
value格式
| Type | value | reserve | cdate | timestamp |
| 1B | | 16B | 8B | 8B |
在解析的时候,先解析头部的第一个字节,然后根据类型判断是否需要继续解析下去
Cmd层多加了一次对返回值的判断(用于判断多key的错误信息)
else if (s_.ToString() == ErrTypeMessage) {
res_.SetRes(CmdRes::kMultiKey);
}
Cache层删除了Redis前缀
// 修改前:
std::string CachePrefixKeyK = PCacheKeyPrefixK + key_;
db_->cache()->SetBitIfKeyExist(CachePrefixKeyK, bit_offset_, on_);
// 修改后
db_->cache()->SetBitIfKeyExist(key_, bit_offset_, on_);
Storage层对Expire,Del,Exists,Expireat,Persist,TTL,GetType这几个之前需要遍历每种数据类型的接口修改成指定操作某一个具体类型的接口
修改前:
// Keys Commands
int32_t Storage::Expire(const Slice& key, int64_t ttl, std::map<DataType, Status>* type_status) {
type_status->clear();
int32_t ret = 0;
bool is_corruption = false;
auto& inst = GetDBInstance(key);
// Strings
Status s = inst->StringsExpire(key, ttl);
if (s.ok()) {
ret++;
} else if (!s.IsNotFound()) {
is_corruption = true;
(*type_status)[DataType::kStrings] = s;
}
// Hash
s = inst->HashesExpire(key, ttl);
if (s.ok()) {
ret++;
} else if (!s.IsNotFound()) {
is_corruption = true;
(*type_status)[DataType::kHashes] = s;
}
// Sets
s = inst->SetsExpire(key, ttl);
if (s.ok()) {
ret++;
} else if (!s.IsNotFound()) {
is_corruption = true;
(*type_status)[DataType::kSets] = s;
}
// Lists
s = inst->ListsExpire(key, ttl);
if (s.ok()) {
ret++;
} else if (!s.IsNotFound()) {
is_corruption = true;
(*type_status)[DataType::kLists] = s;
}
// Zsets
s = inst->ZsetsExpire(key, ttl);
if (s.ok()) {
ret++;
} else if (!s.IsNotFound()) {
is_corruption = true;
(*type_status)[DataType::kZSets] = s;
}
if (is_corruption) {
return -1;
} else {
return ret;
}
}
修改后
// Keys Commands
int32_t Storage::Expire(const Slice& key, int64_t ttl) {
auto& inst = GetDBInstance(key);
Status s = inst->Expire(key, ttl);
if (s.ok()) {
return 1;
} else if (!s.IsNotFound()) {
return -1;
}
return 1;
}
Redis层每次需要对Key进行修改时,新增了对meta-key类型的判断
if (!ExpectedMetaValue(Type::kSet, meta_value)) {
return Status::InvalidArgument("WRONGTYPE Operation against a key holding the wrong kind of value");
}
todo: slot migrate 支持 stream
Bot detected the issue body's language is not English, translate it automatically.
todo: slot migrate support stream
补充下新架构的图片吧,在 shaoyi 的图片基础之上进行修改
Bot detected the issue body's language is not English, translate it automatically.
Let’s add pictures of the new architecture and modify them based on shaoyi’s pictures.