valkey-io
Core: Dedicated "connection" pooling
Describe the feature
Currently the core client multiplexes commands over a single connection to the server. However, there are scenarios where an app needs a dedicated connection to the server for correctness or performance reasons.
The app developer could maintain a pool of glide clients themselves, but this requires work on their part and suffers from unnecessary overhead:
- each client tracks topology updates
- each client might keep connections to every node even if commands (e.g. transactions) will be sent to a single node
Use Case
I can think of at least a few cases where this might come in useful:
- Using
WATCHcorrectly. Without a dedicated connection any other concurrent client usage from a different thread may pollute the conditional transaction tracking that the server performs. - Sending/retrieving large payloads. Without a dedicated connection, other threads would be blocked until the payload gets sent over the wire. (Keep me honest, but I assume Valkey's multithreading allows multiple clients to send/receive data concurrently)
- Client-side caching. A dedicated connection could ensure that only the specific keys the application needs are tracked for client-side invalidation (Keep me honest, maybe there's a separate mechanism in which Glide achieves this?)
Proposed Solution
App developers would obtain a "dedicated" client handle from their main client, e.g. (using Go)
dedicatedClient := client.UsingDedicated()
defer dedicatedClient.Close() // release connections once no longer needed
value := dedicatedClient.Get("very-large-key")
Internally, the instance returned from UsingDedicated would contain a handle to a dedicated connection set in the core client. Core client would maintain a map of handles to sets of 0 or more node connections.
Once the app begins sending commands to the dedicated client, core client would obtain 1 or more connections from a connection pool for each shard that is referenced by the command and reuse those connections to execute any subsequent commands (Note: this optimizes the case where a dedicated client is used to send commands to only a subset of shards that make up a cluster).
When the dedicated client is closed/released, core client would return any connections mapped to the handle back to the connection pool.
This is related to the concept outlined in #4035 of multiple glide clients being able to share a single core client.
Other Information
No response
Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
Client version used
latest
Environment details (OS name and version, etc.)
all
@marcind Thanks for opening it, that indeed a wanted feature. The “how” exactly, needs to be designed to be efficient and to support all kinds of use cases. I think it makes sense to merge both issues into one and have the design of both together to see what we can reuse and what is entirely different. If possible, please share your specific use case for which it needed — on top of the general idea and use cases which you thought of, and I invite you to take part in the design meetings to be able to give input in real life.
@khadrawy If this is something you would be interested in as well, after our chat, please share, by a plus one, and if possible your use case. We will benefit from knowing what's user's needs, so we can understand the priority of things.
@avifenesh Thanks for tagging me here. Indeed this would be a beneficial feature 💯
@avifenesh my specific use case is about handling CAS transactions correctly. I'm still unclear on how that's meant to be handled with the Batch concept, and the guidance I've seen so far is to create a new client for every such use case. The app I work on performs a few thousand CAS transactions a second so I care about this being handled efficiently.
I feel that this might solve an issue we are facing. That is we run a cluster with 2 replicas per shard. However we currently have to manually manage connections to the replicas for certain operations like hscan since the internal connection logic may switch which replica subsequent requests with an updated cursor are directed to which causes problems because cursors are not portable between replicas.
Sorry that this feature didn't make it to version 2.1.
All 2.2 candidate will be re-evaluated once we released 2.1.
We tried to make the switch from jedis to valkey-glide but the performance regression was so bad (around 25 times worse for average put latency) that we are going to have to hold off and potentially explore other valkey clients (like valkey-java) if nothing changes here. We probably have a bit of unique case in which we push large files (up to 5 MB) with amazing success and performance, and I'm guessing we are getting hit by what's described in https://github.com/valkey-io/valkey-glide/wiki/General-Concepts about reading and writing large files.
