pydis

pydis is an experiment to disprove some of the falsehoods about performance and optimisation regarding software and interpreted languages in particular.

Below you will find a Redis clone, pydis, written in ~250 lines of idiomatic Python code, providing a subset of redis' functionality for which there are official benchmarks.

Briefly, pydis is ~50% as fast as Redis measured in number operations per second.

P.S. This is not a criticism of Redis, which is a brilliant project and a system-level software that powers thousands of infrastructures. It just happened to be one of the fastest software I could imagine and clone the same day.

Disclaimer

I have used the following libraries written in C for performance:

• uvloop

  uvloop is a fast, drop-in replacement of the built-in asyncio event loop. uvloop is implemented in Cython and uses libuv under the hood. >

• hiredis

  Python extension that wraps protocol parsing code in hiredis.

Discussion

• Hacker News

  1. 2020-11-15 -- Pydis – Redis clone in 250 lines of Python, for performance comparison
  2. 2019-03-02 -- Pydis: Redis clone in Python 3 to make points about performance

• Reddit

  1. 2019-03-01 -- pydis - A redis clone in Python 3 to disprove some falsehoods about performance

Results

redis-benchmark -q -t set,get,incr,lpush,rpush,lpop,rpop,sadd,hset,spop,lrange,mset -n 100000 -P 5 

• 100,000 requests in total per command.
• Requests are pipelined in groups of 5.

Host System

• Ubuntu 20.04
• Python 3.8.5 (GCC 9.3.0)
• Redis 5.0.7 malloc=jemalloc-5.2.1 bits=64 build=636cde3b5c7a3923

Contributions

Contributions are very welcome, given that they fall into one of the following categories:

• Those that improve the performance.
  • The aim of this exercise is to prove that interpreted languages can be just as fast as C. So whilst using a faster parser in C with Python bindings is okay, rewriting pydis in Cython is not.
  • I will accept "minor" deviations from idioms only if the performance gains are worth it; stick to idiomatic Python otherwise!
• Those to achieve feature parity with Redis for which there are official benchmarks.
  • We are not trying to develop a full-featured Redis clone here so please do not implement commands for which there are no official benchmarks.
• Those that fix formatting etc.
  • Please do not invent your own style, use PEP 8.