go-maglev icon indicating copy to clipboard operation
go-maglev copied to clipboard

Generate permutations on the fly during table population

Open wdamron opened this issue 5 years ago • 0 comments

Permutations don't actually need to be materialized before a Table is populated. Instead, they can be computed on-the-fly, which reduces the memory overhead quite a bit.

Additionally, rebuilding a table in multiple steps can minimize the number of reassigned partitions. First, partitions are assigned as normal (i.e. without reassigning partitions from "dead" nodes to "live" nodes). Next, all partitions belonging to "dead" nodes are unassigned. Finally, all unassigned partitions are reassigned (round-robin) amongst the "live" nodes.

This commit changes the Table structure as well as the return type of the Lookup method (which now returns the assigned name rather than its index). This is probably an unwarranted API change, so I don't really expect this PR to be merged; I'm just putting this out there in case you'd be interested in some of the modifications.

wdamron avatar Jul 07 '18 18:07 wdamron