Support multiple shared connections for Redis standalone mode in `LettuceConnectionFactory`

[yangty89]

Hi @mp911de, after having introduced an enhancement for Lettuce with your help, I'm currently looking at a potential performance improvement for Lettuce connection usages managed by Spring :) The general idea is to support multiple shared connections for Redis standalone mode.

By default, LettuceConnectionFactory maintains a single shared connection and will inject it into a newly created LettuceConnection instance each time the factory's getConnection method is called. It's a pretty graceful design as it gets rid of the complexity of maintaining a connection pool, and interacts with Redis in a pipelining way when serving multiple threads, which ensures a high level of performance.

Alternatively, under certain circumstances, one can also create and use a connection pool by using the native Lettuce API. A connection pool may bring about positive impacts on performance by taking advantage of the parallel processing capacity of the underlying multi-core processor. However, due to the thread confinement enforced by the connection pool, the pipelining feature mentioned above would not be well exploited.

I'm therefore thinking about a possibly new pattern of connection usage, which may take advantage of both the pipelining feature of Lettuce and the parallel processing capacity of modern processors, and get the most of both worlds. It may look like something showed below:

To verify the idea, I manually built a list of connections, and shared them between business threads. I then conducted some benchmarking of throughput(ops/s) using RedisClientBenchmark#syncSet method, comparing it with the single-shared-connection pattern and the connection pooling, and even with Jedis. The results show that the multiple-shared-connections pattern outperforms the others when it keeps a good balance between the use of the pipelining feature and the parallel processing capacity, by aligning the number of connections with the number of processors (8 in my case).

I think therefore that it might be worth considering supporting the multiple-shared-connections pattern in Spring for Redis standalone mode, as it might bring about a notable performance improvement under heavy load. Specifically, we could perhaps consider maintaining a list of shared connections in LettuceConnectionFactory. Each time the factory's getConnection method is called, a shared connection could be fetched from the list in a round-robin manner. However, we should carefully create these shared connections so that they are attached to different netty EventLoops, and thus benefit from the system's parallel processing capacity. I would like to know your opinion on this, thanks!