consistent-hash
consistent-hash copied to clipboard
Allows specification of weight by range
Allows specification of weight by range rather than by integer value.
Useful when you need to scale up on a production server without interruption/cache busting. Example:
Scaling
- Start up pool with single server and a 0, 100 range.
connection_hasher = RangeHash({'redis://1.1.1.1:6379': [0, 100]})
- A while later, it's running too hot but you'd rather not freeze production. So you can set up Redis replication, wait until it's synced, and then execute:
connection_hasher.add_nodes({'redis://2.2.2.2:6379': [0, 50]})
- Now you're running half of your load to the original server and half to the new one, and you haven't lost anything in terms of consisteny.
- Repeat as you continue scaling.
Over time, you can scale up more or less indefinitely without any downtime.
Failover
You can also handle failover gracefully since hashing is independent of hostname - starting the example above, you'd run connection_hasher.remove_nodes(['redis://2.2.2.2:6379])
then connection_hasher.add_nodes({'redis://3.3.3.3:6379': [0, 50]})
to handle failing over from 2.2.2.2 to 3.3.3.3 with no service interruptions, cache stampede, and only minor loss of consistency.
Includes the other commit, but you might want this addition structured differently, or have it in a separate distribution - LMK if that's the case. Thanks!
I think this feature is not necessary.As both of the range [0,50] and the integer 50 equivalent to xrange(0,50).
I'm not sure what you mean - could you give an example?
I mean that the RangeHash({'redis://1.1.1.1:6379': [0, 100]}) could be instead with the ConsistentHash('redis://1.1.1.1:6379': 100}.Refer to the ‘def _node_keys(self, node)’ routine, they are same.
Because the hashing is independent of hostname, so after connection_hasher.add_nodes({'redis://2.2.2.2:6379': [0, 50]}), the range [0, 50] will hit 2.2.2.2, it will cause mishit the range [0, 50] of the original mapping.