Suggestion: consolidate empty lines in flux resource list's default format

[kkier]

The current/new-in-.65 behavior is to output a full line for each empty queue in flux resource list like so:

[root@elcap1:~]# flux resource list
[root@elcap1:~]# flux resource list
     STATE QUEUE      NNODES   NCORES    NGPUS NODELIST
      free foo             0        0        0 
      free bar             0        0        0 
      free baz	           0        0        0 
      free qux             0        0        0 
      free quux            0        0        0 
 allocated foo             0        0        0 
 allocated bar             0        0        0 
 allocated baz             0        0        0 
 allocated qux             0        0        0 
 allocated quux            0        0        0 
      down foo,qux         1        1        1 elcap[XXX-XXX]
      down foo,bar         2        2        2 elcap[XXX-XXX]
      down foo,baz         3        3        3 elcap[XXX-XXX]
      down foo,quux        4        4        4 elcap[XXX-XXX]

I'm wondering if it might be cleaner to, in the case of lines with 0 nodes associated, use state as a primary indicator and combine the queue list like we do with overlapping queues, so you'd end up with something like:

     STATE QUEUE      NNODES   NCORES    NGPUS NODELIST
      free bar,quux        0        0        0 
      free baz             1        1        1
 allocated foo,bar,baz     0        0        0     

Immediate problem I see is that this causes some conceptual overlap between "the first line above indicates these queues have 0 nodes in this state" and "the first line above indicates that there are 0 nodes in this state who that are also members of both of these queues." I don't think of that as being confusing offhand, but I can see how that might not be universal.

To put it another way, in the example above does this line mean "there are no nodes in queues bar or quux that are free," or does it mean "there are no nodes in both queues bar and quux that are free, but there may be nodes in either of them individually free."

Reading this back, I'm not sure if it makes 100% sense...

[kkier]

To explain a little more simply, I propose something where in the first situation you would end up with this output:

[root@elcap1:~]# flux resource list -o "{state:>10} ?:{queue:<20.20} ?:{propertiesx:<10.10+} {nnodes:>6} {ncores:>8} ?:{ngpus:>8} {nodelist}"
     STATE QUEUE                NNODES   NCORES    NGPUS NODELIST
      free foo,bar,baz,qux,quux      0        0        0 
 allocated foo,bar,baz,qux,quux      0        0        0 
      down foo,qux                   1        1        1 elcap[XXX-XXX]
      down foo,bar                   2        2        2 elcap[XXX-XXX]
      down foo,baz                   3        3        3 elcap[XXX-XXX]
      down foo,quux                  4        4        4 elcap[XXX-XXX]

[grondo]

Note that we don't currently have a way to expand a field to make room for a larger list of queues, so at this point the output would be more like:

     STATE QUEUE      NNODES   NCORES    NGPUS NODELIST
      free foo,bar,ba      0        0        0 
 allocated foo,bar,ba      0        0        0 
      down foo,qux         1        1        1 elcap[XXX-XXX]
      down foo,bar         2        2        2 elcap[XXX-XXX]
      down foo,baz         3        3        3 elcap[XXX-XXX]
      down foo,quux        4        4        4 elcap[XXX-XXX]

Which is probably unhelpful

[kkier]

I don't suppose it's reasonable to increase the default format's {queue} field to length 20? I have no feel for how common overlapping queues are outside if LLNL.

[grondo]

Another approach might be to develop a format specifier or other indicator that sets the width of a field to the minimum width that would not result in the field being truncated on any line. We do already have the ?: prefix, so I'm thinking this could be done. Then we could use that in the default format(s) for flux resource list.

Then we could look into combining the empty sets into one line.

[kkier]

Another approach might be to develop a format specifier or other indicator that sets the width of a field to the minimum width that would not result in the field being truncated on any line. We do already have the ?: prefix, so I'm thinking this could be done. Then we could use that in the default format(s) for flux resource list.

Then we could look into combining the empty sets into one line.

This is a substantially better idea!

[grondo]

Ok, now that #6284 has been merged, we can safely change the behavior of flux resource list to combine all empty sets into one (by adding all queues to one empty set, instead of creating an empty set per queue).

The result goes from this:

     STATE QUEUE      NNODES   NCORES    NGPUS NODELIST
      free all,debug      61     5856      244 fake[0-60]
      free all,test      739    70944     2956 fake[61-799]
      free all,comput  15584  1496064    62336 fake[800-16383]
 allocated all             0        0        0 
 allocated test            0        0        0 
 allocated compute         0        0        0 
 allocated debug           0        0        0 
      down all             0        0        0 
      down test            0        0        0 
      down compute         0        0        0 
      down debug           0        0        0 

to:

     STATE QUEUE                  NNODES  NCORES NGPUS NODELIST
      free all,debug                  61    5856   244 fake[0-60]
      free all,test                  739   70944  2956 fake[61-799]
      free all,compute             15584 1496064 62336 fake[800-16383]
 allocated all,test,compute,debug      0       0     0 
      down all,test,compute,debug      0       0     0 

I'm not sure if this is confusing or not. Maybe @kkier's initial suggestion (offline) to suppress "empty" lines by default would be better? (This is what flux resource status does unless the state is specifically requested)

Happy to go either way here.

[grondo]

Looking back, it is a bit confusing that non-empty sets that overlap are combined (e.g. all,debug) and not the empty set. Though admittedly an empty set is a special case since it has no nodes, the nodelist for all queues matches.

Perhaps it would be good to present empty sets on one line, regardless of whether they are displayed by default or not.