Regroup option to have "any" semantics instead of "all"

[kent-engstrom]

I am investigating if we can use "nodeset" instead of our homegrown "nodes" tool (https://www.nsc.liu.se/~kent/nodes/). I lack one feature that we use a lot with nodes --- the possibility to get all groups that include at least one node referenced by the nodeset.

To explain why we find it useful, imagine that you have cluster hardware consisting of four nodes per chassi. For example, chassis c1 is n[1-4], chassis c2 is n[5-8], etc. Now imagine that some nodes are malfunctioning, and to fix them you need to take down their chassis (also affecting the other nodes), so you want to drain them all. Using the "nodes" tool, I could either do

% ./nodes --up chassis n[3-5,15] 
c[1-2,4]

to get the list of chassis affected, or

% ./nodes --fill chassis n[3-5,15]
n[1-8,13-16]

to map them back to nodes in the same operation.

(BTW, the is also the --gather operation, that is basically the same as --regroup in nodeset.)

It would be nice to have an operation in nodeset that worked like "nodes --up", perhaps something like

% nodeset -s chassis --regroup-any n[3-5,15]
@chassis:c[1-2,4]

Would that be possible?

[thiell]

Hi @kent-engstrom!

One solution for this use-case is to pass a node set as argument of nodeset --list (or -l). So instead of just listing node groups, the command will find node groups these nodes belong to within the specified group source (-s), or within the default group source if not specified. We need to clarify this in the doc though... I didn't even find it in the online doc.

Example config (/etc/clustershell/groups.d/cluster.yaml):

roles:
    compute: 'n[1-8]'

...

chassis:
    c1: 'n[1-4]'
    c2: 'n[5-8]'
    c3: 'n[9-12]'
    c4: 'n[13-16]'

To mimic your nodes --up command, I would either use:

nodeset -s chassis -l n[3-5,15]
@chassis:c1
@chassis:c2
@chassis:c4

or, closer to what you get:

$ nodeset -s chassis -l n[3-5,15] | nodeset -s chassis -r
@chassis:c[1-2,4]
$ nodeset -s chassis -l n[3-5,15] | nodeset -s chassis -G -r
@c[1-2,4]

Note: it would be nice to further think about this use case and possibly find a way to avoid the use of a second command with -r...

To mimic your nodes --fill option, I would use:

$ nodeset -s chassis -l n[3-5,15] | nodeset -f
n[1-8,13-16]

[kent-engstrom]

Thanks for the quick reply! This works as replacements for "nodes --up" and "nodes --fill", but as you said, it would be nice to not have to do the pipelining.

[thiell]

Hi @kent-engstrom,

We've finally reviewed this feature request in details:

• nodes --fill makes sense to be performed using two "base" nodeset commands. Also we would like to keep nodeset as simple as possible. Actually some might already think it's too complicated. :)
• nodes --up: having a way to do something similiar using a single nodeset command is actually a good idea, so I would like to push this new feature into 1.8, but before that, we wanted to get your feedback about the following changes:

CLI/Nodeset: extend --list output to group sets (#345)

Currently the -l,--list and -L,--list-all commands can only **list**
node groups.

Extend the --list and --list-all command options so they can work with
-c/-e/-f to respectively count, expand or fold node groups as "group
sets".

It is important to remember that used like this, --count will
count groups, not nodes:

    $ nodeset -s chassis -lll
    @chassis:c1 n[1-4] 4
    @chassis:c2 n[5-8] 4
    @chassis:c3 n[9-12] 4
    @chassis:c4 n[13-16] 4

    $ nodeset -s chassis -l -c
    4

Standard options that come with -c/-e/-f are supported, like -S:

    $ nodeset -s chassis -l -e -S'\t'
    @chassis:c1	@chassis:c2	@chassis:c3	@chassis:c4

This does also simplify the following pipelining:

    $ nodeset -s chassis -l n[3-5,15] | nodeset -s chassis -r
    @chassis:c[1-2,4]

to just:

    $ nodeset -s chassis -l -f n[3-5,15]
    @chassis:c[1-2,4]

-G/--groupbase is also supported:

    $ nodeset -s chassis -l -f -G n[3-5,15]
    @c[1-2,4]

Feedback welcome :)

The patch is being reviewed at https://review.gerrithub.io/#/c/377642/