gluster
Move the min/max number of nodes in a separate file
I think it would be beneficial to have the min/max number of nodes in a separate file, something like a recommendation with the best settings. I've put the new file under 'resources/settings/recommended_num_nodes.yml'. Not sure if the best place though.
There is probably some additional logic consideration needed to make the min setting adjustable. We currently make some design choices about replica and disperse configurations that necessitate the 4 and 6 minimums, respectively. So the minimums I would not want to move to a config file without other work.
The maximums are, however, more arbitrary (though they need to be appropriate multiples -- of 2 for replica and of 6 for disperse). I think it would make sense to set the maximum in the flavor OEMID file.
May I ask which other work would be needed to move the min/max into an external file? My aim was to not have a fixed setting into the code but move it into a separate place (but still with recommended options).
The logic to create a replica volume expects N x ( 2 + 1 ) volumes in a chained arbiter configuration. The arbiter brick for the first replica pair of nodes 1 and 2 is selected in what I refer to as "plus one" -- meaning that it will always be located on node 3. So we are required to have a third node, and we also expect heterogenous nodes, so that node 3 will also have data bricks. Node 3 therefore needs a replica peer, node 4. Arbiter bricks for every pair other than the first are selected in a "minus one" method -- so the arbiter for the node pair of 3 and 4 will be on 2.
This is currently the only replication method that the code supports, so our minimum node count for replication is 4:
1 + 2 ( + 3 ) + 3 + 4 ( + 2 )
Disperse volumes have a similar problem -- the code currently supports only 4+2 erasure coding, and best practice will be one brick from a disperse set on each node, and therefore the minimum number of nodes is 6
1 + 2 + 3 + 4 [ + 5 ] [ + 6 ]
In order to support lower minimums, we would need to introduce different replication and disperse options accordingly, like replica 3 or replica 2 (though I would argue against ever designing for replica 2).
Because the point of the gluster-colonizer is to be opinionated and offer well-tested design templates, I don't have any motivation to "boil the ocean" and make sure it supports every possible combination of Gluster configurations. In fact, that would be the anthesis of the design intent. If there is a use case argument for something like replica 3, or 2 + 1 with a dedicated arbiter node, then we could adjust to that.
The maximums, though, were added as somewhat arbitrary support limits. Beyond a certain scale, we felt uncomfortable without real testing. Discovery timeouts and one-off node problems could be more common, and we currently don't have good facilities to handle those conditions. This is why the maximums are presently in the code instead of the flavor templates. But I think in the long run they should be in the templates.
On Wed, Aug 29, 2018, 12:52 AM Pier [email protected] wrote:
May I ask which other work would be needed to move the min/max into an external file? My aim was to not have a fixed setting into the code but move it into a separate place (but still with recommended options).
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/gluster/gluster-colonizer/pull/59#issuecomment-416859871, or mute the thread https://github.com/notifications/unsubscribe-auth/AKKcrrdIjPC-zt2ezA63w48TIoyMID9aks5uVkg8gaJpZM4U9Mex .