valkey-io
[NEW] Server event on cluster topology change
The problem/use-case that the feature addresses
Each search query operation must be fanned out to each shard (selecting from primary/replica accordingly). This requires processing of the entire cluster map, which is an expensive process and a substantial amount of the mainthread processing time for a query operation.
Description of the feature
If a notification of a change in the cluster map could be made available to search, then it could easily use a cached copy of the cluster map -- invalidating that cache as needed. Seems like a server event would be the way to go. In other words, if a server event was published any time there was a change in the cluster map.
I guess it's related to https://github.com/valkey-io/valkey/issues/57. We can introduce hooks in appropriate places to send out notification on topology change to modules as well as to clients if required.
Related in the sense that it's the same source event. Though for search, the implementation is easier as the event can be delivered synchronously -- which is a much simpler implementation.