redis-rb
redis-rb copied to clipboard
redis
Improve performance of Ruby connection driver
Uses an offset pointer for the socket buffer instead of mutating it every time we read data.
This improves the performance and memory consumption when reading from large sets.
This is similar to the optimization done in https://github.com/redis-rb/redis-client/commit/2be1b36846eadd27583a699259a670a8805192f9#diff-e2a648012b25aa4906ce7af76ab8f9a9fe28e144e4338b7b2f99e351cbf0cee5R108
I created a branch that implements this as a separate driver and included a benchmark file: https://github.com/engwan/redis-rb/tree/improve-socket-buffer-patch
Results of the benchmark:
➜ ruby -Ilib benchmarking/ruby_patched.rb
Calculating -------------------------------------
Ruby 5.000 i/100ms
Patched Ruby 12.000 i/100ms
-------------------------------------------------
Ruby 55.025 (± 5.5%) i/s - 275.000
Patched Ruby 129.960 (± 1.5%) i/s - 660.000
Comparison:
Patched Ruby: 130.0 i/s
Ruby: 55.0 i/s - 2.36x slower
Calculating -------------------------------------
Ruby 127.683M memsize ( 0.000 retained)
84.990k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Patched Ruby 5.607M memsize ( 2.795k retained)
55.027k objects ( 1.000 retained)
50.000 strings ( 1.000 retained)
Comparison:
Patched Ruby: 5607194 allocated
Ruby: 127682926 allocated - 22.77x more
More context in: https://gitlab.com/gitlab-org/gitlab/-/issues/367857
I'll try to find some time to review this this week end, but I'm wondering if I shouldn't simply kickstart using redis-client in redis-rb rather than duplicate this code.
🎉 I'd be happy to provide Sidekiq benchmarks before and after once this lands. Redis driver overhead is a major bottleneck.
Closing as the master branch now use redis-client under the hood. I'll publish a 5.0 beta tomorrow.