redis-sharding-hs

redis-sharding-hs copied to clipboard

Redis Sharding

http://github.com/kni/redis-sharding-hs

Haskell version of "Redis Sharding" (http://github.com/kni/redis-sharding)

For resharding see http://github.com/kni/redis-sharding.

Redis Sharding with fully support MSET, MSETNX, MGET, DEL commands.

This Lazy version. Strict version of Redis Sharding available also: http://github.com/kni/redis-sharding-hs-strict. ATTENTION. Strict version faster in 4 time for Pipeline mode!

Standard ML version (https://github.com/kni/redis-sharding-sml) is even more productive.

Nota bene

To achieve the best efficiency while sharding data it is especially important to use pipelining by the client. The importance increases according to the number of node for multikeys commands (multi-nodes commands, if more precisely). And it increases twice only in comparison with the usual redis server. :-)

Build

ghc -threaded -rtsopts -O2 --make redis_sharding.hs
or
cabal configure && cabal build

Run

./redis_sharding --nodes=10.1.1.1:6380,10.1.1.1:6381,...

or to use all CPU core run as

./redis_sharding --nodes=10.1.1.1:6380,10.1.1.1:6381,... +RTS -N

ATTENTION! +RTS -N must be after of all others parameters.

Others parameters:

--host=10.1.1.1
--port=6379
--timeout=300 (0 - disable timeout)

Tuning

./redis_sharding --nodes=10.1.1.1:6380,10.1.1.1:6381,... +RTS -N -A10M -qa

Bug

Do not use too long keys (and values) or the excessive number of keys in one command (>100Kb). GHC threaded runtime contains an error that causes data loss of a socket status that it's ready for write. Not-threaded runtime does not have the above mentioned bug but it is slower and uses the system select call instead of using kevent or epoll. Tests showed that this error is present in GHC 7.8.[34] and 8.0.1.

Notes

timeout

As opposed to the Perl version, the Haskell version RedisSharding is built in the "lazy style".
If the client is not responding, RedisSharding won't detect closing connection by the server because of time-out.
So, the time-outs execution is added into the RedisSharding itself (timeout keys, seconds).