jedis
jedis copied to clipboard
redis
Support non-blocking (Async) APIs
On #647, we switched I/O from Stream based Old I/O to New I/O. Based on New I/O, finally I can introduce non-blocking APIs to Jedis!
I'm not familiar with Java New I/O, and our clients may have Java 6, so there're some restrictions on implementation. I'm trying to implement it simply, so I try to avoid multi-thread things as many as possible. (It may lose some performances.) 1 dispatcher handles 1 connection's request/response with only one thread.
Current APIs doesn't support Sharded, Redis Cluster, Sentinel, scripting, transaction, pipeline. You can't select DB index, and you should present AUTH password with AsyncJedis constructor.
I wish NIO experts would improve my base codes to gain high performance.
Please review and comment. Thanks!
ps. It's upmerged. ps2. Failing test is not related to async, it's related to https://github.com/antirez/redis/issues/1561. We should fix it by create another issue.
This change is 
Performance test by benchmark code (run 5 times)
Sync
33534 33008 32190 33311 33489
-> 33106.4
Async
59435 38431 44072 39370 35174
—> 43296.4
@HeartSaVioR excellent work!. What do the numbers in the benchmark mean?.
@marcosnils Thanks! Btw, benchmark number means ops (operations per second). You can reference it by https://github.com/xetorthio/jedis/blob/master/src/test/java/redis/clients/jedis/tests/benchmark/GetSetBenchmark.java (blocking version)
Btw, callback should return ASAP because its thread is dispatcher thread. We can introduce callback worker thread or pool if it really make some problems.
I'm still in the process of reviewing the code, specifically commit b1e019c0e9e393cb1100c5ce44391f2f7503e1dc (NIO changes).
The main question I have for you @HeartSaVioR and for the rest of the reviewers and community is about the use of callbacks versus the use of futures. This is a design decision that will affect the users of Jedis that use the async API.
I have an inclination towards using futures since I have been making use of them for the past several years when I build my applications. I find they allow for a cleaner more streamlined design compared to callbacks; however, that's my opinion, and I'm interested in what others have to say (=
@mardambey We can gather users' current opinions of async. :)
- #241
- https://groups.google.com/forum/#!topic/jedis_redis/HfplRrc1KNQ
Personally I've selected callback, because it can be collaborated with Java 8 lambda. And recently "event-driven" is something hot, so we can gain advantage when we support callback. ;) But we can add Future<?> to return type, and support both type. (As scala-redis-nb is, but it's debug purpose. please see https://github.com/debasishg/scala-redis-nb/blob/master/README.md#sample-usage)
I wish someone crafts long-running benchmark or long-running tests report. :)
Here is @rdifalco 's message from private email:
I noticed I could improve the NIO benchmark if I used a ByteBuffer.allocate instead of a ByteBuffer.allocateDirect. This is primarily because the JVM must cross the JNI boundary for every .get and every .put. And for Jedis reading A LOT of these need to be executed to read until '/r/n'.
You might want to try it yourself. It's possible that you could allocateDirect for the Writer, but I think the reader will always perform better with a Heap based ByteBuffer.
I also think it's valuable to give it a try.
Btw, we're reviewing #780 to get a higher performance, which could be also applied to current PR.
upmerged to current master
In this PR, unit tests takes long time, which I wish to address soon.
After profiling with Yourkit, I can find current implementation consumes lots of memory so there is huge performance drops with long running benchmark. I'm trying to reduce memory consumption.
Tuned allocation telemetry is here.
Allocation telemetry
+-----------------------------------------------------------------------------------------------------------------------------------------+---------------------+------------------------+
| Name | Recorded Objects | Size |
+-----------------------------------------------------------------------------------------------------------------------------------------+---------------------+------------------------+
| +---<All threads> | 37,247,565 100 % | 1,261,812,024 100 % |
| | | | |
| +---com.intellij.rt.execution.application.AppMain.main(String[]) | 20,716,021 56 % | 739,202,616 59 % |
| | | | | |
| | +---NativeMethodAccessorImpl.java (native) redis.clients.jedis.tests.benchmark.AsyncGetSetBenchmark.main(String[]) | 20,716,021 56 % | 739,202,616 59 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:34 redis.clients.jedis.async.AsyncJedis.set(AsyncResponseCallback, String, String) | 9,397,476 25 % | 338,382,896 27 % |
| | | | | | |
| | | +---AsyncJedis.java:282 redis.clients.jedis.async.AsyncBinaryJedis.set(AsyncResponseCallback, byte[], byte[]) | 6,961,047 19 % | 232,674,240 18 % |
| | | | | | | |
| | | | +---AsyncBinaryJedis.java:342 redis.clients.jedis.async.request.RequestBuilder.build(Protocol$Command, byte[][]) | 4,873,075 13 % | 179,734,016 14 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:16 redis.clients.jedis.async.request.RequestBuilder.build(byte[], byte[][]) | 4,873,075 13 % | 179,734,016 14 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:38 redis.clients.jedis.async.request.RequestBuilder.writeIntCrLf(ByteArrayOutputStream, int) | 1,569,559 4 % | 57,340,032 5 % |
| | | | | | | | | |
| | | | | | +---RequestBuilder.java:95 | 1,218,298 3 % | 29,239,152 2 % |
| | | | | | | | | |
| | | | | | +---RequestBuilder.java:114 java.io.ByteArrayOutputStream.write(byte[], int, int) | 351,261 1 % | 28,100,880 2 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:43 java.io.ByteArrayOutputStream.toByteArray() | 867,035 2 % | 55,490,240 4 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:32 redis.clients.jedis.async.request.RequestBuilder.writeIntCrLf(ByteArrayOutputStream, int) | 866,994 2 % | 20,807,856 2 % |
| | | | | | | | | |
| | | | | | +---RequestBuilder.java:95 | 866,994 2 % | 20,807,856 2 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:28 | 866,915 2 % | 20,805,960 2 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:30 redis.clients.jedis.async.request.RequestBuilder.writeIntCrLf(ByteArrayOutputStream, int) | 351,397 1 % | 8,433,528 1 % |
| | | | | | | | | |
| | | | | | +---RequestBuilder.java:95 | 351,397 1 % | 8,433,528 1 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:28 java.io.ByteArrayOutputStream.<init>() | 351,175 1 % | 16,856,400 1 % |
| | | | | | | |
| | | | +---AsyncBinaryJedis.java:343 redis.clients.jedis.async.process.AsyncDispatcher.registerRequest(AsyncJedisTask) | 869,556 2 % | 20,888,352 2 % |
| | | | | | | | |
| | | | | +---AsyncDispatcher.java:119 java.util.concurrent.LinkedBlockingDeque.add(Object) | 869,556 2 % | 20,888,352 2 % |
| | | | | | | |
| | | | +---AsyncBinaryJedis.java:342 | 867,180 2 % | 20,812,320 2 % |
| | | | | | | |
| | | | +---AsyncBinaryJedis.java:343 | 351,236 1 % | 11,239,552 1 % |
| | | | | | |
| | | +---AsyncJedis.java:282 redis.clients.util.SafeEncoder.encode(String) | 2,436,429 7 % | 105,708,656 8 % |
| | | | | | |
| | | +---SafeEncoder.java:26 java.lang.String.getBytes(String) | 2,436,429 7 % | 105,708,656 8 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:35 redis.clients.jedis.async.AsyncJedis.get(AsyncResponseCallback, String) | 6,445,212 17 % | 214,498,152 17 % |
| | | | | | |
| | | +---AsyncJedis.java:219 redis.clients.jedis.async.request.RequestBuilder.build(Protocol$Command, byte[][]) | 3,654,814 10 % | 116,951,376 9 % |
| | | | | | | |
| | | | +---RequestBuilder.java:16 redis.clients.jedis.async.request.RequestBuilder.build(byte[], byte[][]) | 3,654,814 10 % | 116,951,376 9 % |
| | | | | | | |
| | | | +---RequestBuilder.java:38 redis.clients.jedis.async.request.RequestBuilder.writeIntCrLf(ByteArrayOutputStream, int) | 867,119 2 % | 20,810,856 2 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:95 | 867,119 2 % | 20,810,856 2 % |
| | | | | | | |
| | | | +---RequestBuilder.java:28 java.io.ByteArrayOutputStream.<init>() | 867,031 2 % | 41,617,488 3 % |
| | | | | | | |
| | | | +---RequestBuilder.java:30 redis.clients.jedis.async.request.RequestBuilder.writeIntCrLf(ByteArrayOutputStream, int) | 866,904 2 % | 20,805,696 2 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:95 | 866,904 2 % | 20,805,696 2 % |
| | | | | | | |
| | | | +---RequestBuilder.java:28 | 351,361 1 % | 8,432,664 1 % |
| | | | | | | |
| | | | +---RequestBuilder.java:32 redis.clients.jedis.async.request.RequestBuilder.writeIntCrLf(ByteArrayOutputStream, int) | 351,270 1 % | 8,430,480 1 % |
| | | | | | | | |
| | | | | +---RequestBuilder.java:95 | 351,270 1 % | 8,430,480 1 % |
| | | | | | | |
| | | | +---RequestBuilder.java:43 java.io.ByteArrayOutputStream.toByteArray() | 351,129 1 % | 16,854,192 1 % |
| | | | | | |
| | | +---AsyncJedis.java:219 redis.clients.util.SafeEncoder.encode(String) | 1,218,328 3 % | 52,859,160 4 % |
| | | | | | | |
| | | | +---SafeEncoder.java:26 java.lang.String.getBytes(String) | 1,218,328 3 % | 52,859,160 4 % |
| | | | | | |
| | | +---AsyncJedis.java:220 | 867,185 2 % | 27,749,920 2 % |
| | | | | | |
| | | +---AsyncJedis.java:220 redis.clients.jedis.async.process.AsyncDispatcher.registerRequest(AsyncJedisTask) | 353,721 1 % | 8,509,760 1 % |
| | | | | | | |
| | | | +---AsyncDispatcher.java:119 java.util.concurrent.LinkedBlockingDeque.add(Object) | 353,721 1 % | 8,509,760 1 % |
| | | | | | |
| | | +---AsyncJedis.java:219 | 351,164 1 % | 8,427,936 1 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:34 java.lang.StringBuilder.toString() | 1,218,394 3 % | 43,113,360 3 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:33 java.lang.StringBuilder.toString() | 1,218,355 3 % | 43,115,400 3 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:33 java.lang.StringBuilder.<init>() | 867,035 2 % | 41,617,680 3 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:34 java.lang.StringBuilder.<init>() | 866,915 2 % | 41,611,920 3 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:34 | 351,397 1 % | 8,433,528 1 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:33 | 351,236 1 % | 8,429,664 1 % |
| | | | | |
| | +---AsyncGetSetBenchmark.java:38 redis.clients.jedis.async.AsyncBinaryJedis.close() | 1 0 % | 16 0 % |
| | | | | |
| | +---AsyncBinaryJedis.java:52 java.lang.ClassLoader.loadClass(String) | 1 0 % | 16 0 % |
| | | | |
| +---redis.clients.jedis.async.process.AsyncDispatcher.run() | 16,531,543 44 % | 522,609,384 41 % |
| | | | | |
| | +---AsyncDispatcher.java:100 redis.clients.jedis.async.process.AsyncDispatcher.handleRead(SelectionKey) | 14,580,578 39 % | 475,698,576 38 % |
| | | | | | |
| | | +---AsyncDispatcher.java:224 redis.clients.jedis.async.process.AsyncJedisTask.appendPartialResponse(byte) | 10,860,188 29 % | 386,409,216 31 % |
| | | | | | | |
| | | | +---AsyncJedisTask.java:54 redis.clients.jedis.async.response.BasicResponseBuilder.appendPartialResponse(byte) | 9,744,793 26 % | 350,716,576 28 % |
| | | | | | | | |
| | | | | +---BasicResponseBuilder.java:17 | 3,720,390 10 % | 178,578,720 14 % |
| | | | | | | | |
| | | | | +---BasicResponseBuilder.java:29 redis.clients.jedis.async.response.SimpleStringResponseBuilder.appendPartialResponse(byte) | 3,720,390 10 % | 89,289,360 7 % |
| | | | | | | | | |
| | | | | | +---SimpleStringResponseBuilder.java:9 java.lang.StringBuilder.toString() | 3,720,390 10 % | 89,289,360 7 % |
| | | | | | | | |
| | | | | +---BasicResponseBuilder.java:29 redis.clients.jedis.async.response.BulkStringResponseBuilder.appendPartialResponse(byte) | 1,139,795 3 % | 27,550,280 2 % |
| | | | | | | | | |
| | | | | | +---BulkStringResponseBuilder.java:13 redis.clients.jedis.async.response.BulkStringResponseBuilder.handleLength(byte) | 1,139,795 3 % | 27,550,280 2 % |
| | | | | | | | | |
| | | | | | +---BulkStringResponseBuilder.java:42 java.lang.StringBuilder.toString() | 1,115,395 3 % | 26,769,480 2 % |
| | | | | | | | | |
| | | | | | +---BulkStringResponseBuilder.java:50 | 24,400 0 % | 780,800 0 % |
| | | | | | | | |
| | | | | +---BasicResponseBuilder.java:23 redis.clients.jedis.async.response.SimpleStringResponseBuilder.<init>() | 1,139,795 3 % | 54,124,560 4 % |
| | | | | | | | | |
| | | | | | +---SimpleStringResponseBuilder.java:5 java.lang.StringBuilder.<init>() | 1,115,395 3 % | 53,538,960 4 % |
| | | | | | | | | |
| | | | | | +---SimpleStringResponseBuilder.java:5 | 24,400 0 % | 585,600 0 % |
| | | | | | | | |
| | | | | +---BasicResponseBuilder.java:17 redis.clients.jedis.async.response.BulkStringResponseBuilder.<init>() | 24,402 0 % | 1,171,272 0 % |
| | | | | | | | | |
| | | | | | +---BulkStringResponseBuilder.java:7 java.lang.StringBuilder.<init>() | 24,401 0 % | 1,171,248 0 % |
| | | | | | | | | |
| | | | | | +---BulkStringResponseBuilder.java:7 | 1 0 % | 24 0 % |
| | | | | | | | |
| | | | | +---BasicResponseBuilder.java:17 java.lang.ClassLoader.loadClass(String) | 21 0 % | 2,384 0 % |
| | | | | | | |
| | | | +---AsyncJedisTask.java:54 redis.clients.jedis.async.process.AsyncJedisTask.getResponseBuilder() | 1,115,395 3 % | 35,692,640 3 % |
| | | | | | | |
| | | | +---AsyncJedisTask.java:81 redis.clients.jedis.async.process.AsyncJedisTask.initializeResponseBuilder() | 1,115,395 3 % | 35,692,640 3 % |
| | | | | | | |
| | | | +---AsyncJedisTask.java:50 | 1,115,395 3 % | 35,692,640 3 % |
| | | | | | |
| | | +---AsyncDispatcher.java:227 redis.clients.jedis.async.process.AsyncJedisTask.callback() | 3,720,390 10 % | 89,289,360 7 % |
| | | | | | |
| | | +---AsyncJedisTask.java:69 redis.clients.jedis.BuilderFactory$5.build(Object) | 3,720,390 10 % | 89,289,360 7 % |
| | | | | | |
| | | +---BuilderFactory.java:56 redis.clients.jedis.BuilderFactory$5.build(Object) | 3,720,390 10 % | 89,289,360 7 % |
| | | | | | |
| | | +---BuilderFactory.java:58 redis.clients.util.SafeEncoder.encode(byte[]) | 3,720,390 10 % | 89,289,360 7 % |
| | | | | | |
| | | +---SafeEncoder.java:34 | 3,720,390 10 % | 89,289,360 7 % |
| | | | | |
| | +---AsyncDispatcher.java:104 redis.clients.jedis.async.process.AsyncDispatcher.handleWrite(SelectionKey) | 1,950,962 5 % | 46,910,728 4 % |
| | | | | | |
| | | +---AsyncDispatcher.java:201 java.util.concurrent.LinkedBlockingDeque.add(Object) | 1,940,007 5 % | 46,560,168 4 % |
| | | | | | |
| | | +---AsyncDispatcher.java:200 java.util.concurrent.LinkedBlockingDeque.poll() | 6,058 0 % | 193,856 0 % |
| | | | | | |
| | | +---AsyncDispatcher.java:202 java.util.concurrent.LinkedBlockingDeque.peek() | 4,897 0 % | 156,704 0 % |
| | | | | |
| | +---AsyncDispatcher.java:105 sun.nio.ch.SelectionKeyImpl.interestOps(int) | 2 0 % | 48 0 % |
| | | | | |
| | +---AsyncDispatcher.java:88 sun.nio.ch.SelectorImpl.select(long) | 1 0 % | 32 0 % |
| | | | |
| +---java.lang.ref.Finalizer$FinalizerThread.run() | 1 0 % | 24 0 % |
+-----------------------------------------------------------------------------------------------------------------------------------------+---------------------+------------------------+
Generated by YourKit Java Profiler 2014 build 14108 Dec 14, 2014 3:26:32 PM
Three major things have been left.
- building request (into byte[])
- building response (apply each byte of chunk of Redis response)
- enqueue / dequeue item
Any contribution on this is more welcome! :)
sync
34000 ops 34000 ops 34000 ops
async
146000 ops 142000 ops 152000 ops
pipeline
544000 ops 546000 ops 558000 ops
Async is faster than previous, but pipeline is still more faster than async.
I've tuned RequestBuilder once more to reduce array allocation. During long-live benchmark (AsyncGetSetBenchmark) there're no major GC, most of generated objects by async feature are in Eden. We can now concentrate on optimizing building responses.
We can apply RequestBuilder to implement Transaction to JedisCluster, since pipelining strategy doesn't work with redirection so we should also queue request itself.
I've modified about building responses to let AsyncDispatcher passes ByteBuffer to ResponseBuilder, and let ResponseBuilder takes care of it. It seems not increase noticeable performance, but I hope that will be optimized based on this.
Seems like we're ready to review and adopt it to 3.0 after reviewing. :D
I've just added support of Lua Script from AsyncJedis. Seems like I've added all features I can handle with async. (Transaction, Cluster, Sentinel seems not fit asynchronously)
@HeartSaVioR amazing work. I'll try to give it a look as fast as possible so we can release something to the community.
Upmerged to current master.
I revised current benchmark and ran to compare Normal vs Async vs Pipeline. Please refer https://groups.google.com/d/msg/jedis_redis/7ZJPFp_qTY0/VVY3p1D5EzcJ for details.
Nils Kilden-Pedersen pointed out from Google Groups.
https://groups.google.com/d/msg/jedis_redis/7ZJPFp_qTY0/Cy_09rnCqMkJ
- Method AsyncDispatcher.registerRequest is synchronized. Since it only calls LinkedBlockingDeque.add and Selector.wakeup that should be unnecessary, as both are safe to call concurrently, and I don’t believe those two method calls need to be atomic.
- Same class, method handleConnectionException, while loop does a peek then a poll, which is redundant, since poll returns null if empty. Probably not super important since it’s during an exceptional condition, but still…
- Java7 introduced ConcurrentLinkedDeque, which is a non-blocking deque implementation. This makes it better than LinkedBlockingDeque under load, but is supposedly slightly less performant in the non-contended case. I think for this scenario, ConcurrentLinkedDeque offers better scalability.
I'll try to adopt these one by one and check it would help.
We're talking about changing our async implementation to Netty, but it's not confirmed so I'll try both version of async, NIO and Netty.
Compared to using Netty, lack of ReplayingDecoder we need to handle response by struggling with byte. When we decide to stick NIO, we need to implement custom ReplayingDecoder and let ResponseHandler simplified.
@HeartSaVioR :+1: Any ETA on this pull request getting merged to master ?
Is this still a priority to add Async apis to Jedis? Currently looking a Lettuce for this reason alone, although I prefer the lightweight Jedis.
@HeartSaVioR sorry to bother you, do you happen to know if there are still any plans to support async features in jedis?
