feat: When using redis cluster for caching, add global_prefix to all keys cached by redis
Summary
Fixes #13604 Add REDIS_KEY_PREFIX configuration, default is empty
[!Tip] Close issue syntax:
Fixes #<issue number>orResolves #<issue number>, see documentation for more details.
Screenshots
| Before | After |
|---|---|
| ... |
Checklist
[!IMPORTANT]
Please review the checklist below before submitting your pull request.
- [ ] This change requires a documentation update, included: Dify Document
- [x] I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
- [x] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
- [x] I've updated the documentation accordingly.
- [x] I ran
dev/reformat(backend) andcd web && npx lint-staged(frontend) to appease the lint gods
After using these changes, when my Redis cluster is version 6.0 and ACL user management is enabled, I encounter some issues. For example, the Redis cluster must enter the account and password at the same time to log in. At the same time, an error occurs when executing the redis_client.lock code, indicating that the current user has no permission to execute the command. I have solved the relevant issues. How can I submit the solution?
my solution is :
1、/api/extensions/ext_redis.py
elif dify_config.REDIS_USE_CLUSTERS:
assert dify_config.REDIS_CLUSTERS is not None, "REDIS_CLUSTERS must be set when REDIS_USE_CLUSTERS is True"
nodes = [
ClusterNode(host=node.split(":")[0], port=int(node.split(":")[1]))
for node in dify_config.REDIS_CLUSTERS.split(",")
]
# FIXME: mypy error here, try to figure out how to fix it
username=dify_config.REDIS_CLUSTERS_USERNAME
if username:
redis_client.initialize(
RedisCluster(startup_nodes=nodes, password=dify_config.REDIS_CLUSTERS_PASSWORD, username=dify_config.REDIS_CLUSTERS_USERNAME),
prefix=dify_config.REDIS_KEY_PREFIX,
) # type: ignore
else:
redis_client.initialize(
RedisCluster(startup_nodes=nodes, password=dify_config.REDIS_CLUSTERS_PASSWORD),
prefix=dify_config.REDIS_KEY_PREFIX,
) # type: ignore
2、/api/commands.py
@click.command("upgrade-db", help="Upgrade the database")
def upgrade_db():
click.echo("Preparing database migration...")
lock = redis_client.lock(name=f"{dify_config.REDIS_KEY_PREFIX}:db_upgrade_lock", timeout=60)
if lock.acquire(blocking=False):
3、/api/services/dataset_service.py
there are four positions that need to change in this py-file
lock_name = "{}:add_document_lock_dataset_id_{}".format(dify_config.REDIS_KEY_PREFIX, dataset.id)
with redis_client.lock(lock_name, timeout=600):
Hello sorry for the late response, could you resolve the conflicts and trigger it again?
@jiandanfeng Hello could you resolve the conflict again?