consistent-hash icon indicating copy to clipboard operation
consistent-hash copied to clipboard

Allows specification of weight by range

Open combatpoodle opened this issue 11 years ago • 4 comments

Allows specification of weight by range rather than by integer value.

Useful when you need to scale up on a production server without interruption/cache busting. Example:

Scaling

  1. Start up pool with single server and a 0, 100 range. connection_hasher = RangeHash({'redis://1.1.1.1:6379': [0, 100]})
  2. A while later, it's running too hot but you'd rather not freeze production. So you can set up Redis replication, wait until it's synced, and then execute: connection_hasher.add_nodes({'redis://2.2.2.2:6379': [0, 50]})
  3. Now you're running half of your load to the original server and half to the new one, and you haven't lost anything in terms of consisteny.
  4. Repeat as you continue scaling.

Over time, you can scale up more or less indefinitely without any downtime.

Failover

You can also handle failover gracefully since hashing is independent of hostname - starting the example above, you'd run connection_hasher.remove_nodes(['redis://2.2.2.2:6379]) then connection_hasher.add_nodes({'redis://3.3.3.3:6379': [0, 50]}) to handle failing over from 2.2.2.2 to 3.3.3.3 with no service interruptions, cache stampede, and only minor loss of consistency.

Includes the other commit, but you might want this addition structured differently, or have it in a separate distribution - LMK if that's the case. Thanks!

combatpoodle avatar Feb 03 '14 02:02 combatpoodle

I think this feature is not necessary.As both of the range [0,50] and the integer 50 equivalent to xrange(0,50).

yummybian avatar Mar 02 '14 13:03 yummybian

I'm not sure what you mean - could you give an example?

combatpoodle avatar Mar 05 '14 19:03 combatpoodle

I mean that the RangeHash({'redis://1.1.1.1:6379': [0, 100]}) could be instead with the ConsistentHash('redis://1.1.1.1:6379': 100}.Refer to the ‘def _node_keys(self, node)’ routine, they are same.

yummybian avatar Mar 18 '14 15:03 yummybian

Because the hashing is independent of hostname, so after connection_hasher.add_nodes({'redis://2.2.2.2:6379': [0, 50]}), the range [0, 50] will hit 2.2.2.2, it will cause mishit the range [0, 50] of the original mapping.

yummybian avatar Oct 07 '14 06:10 yummybian