Add partition information to block device collector output and analyser

[banjoh]

Describe the rationale for the suggested feature.

Some add-ons such as the rook add-on require an unpartitioned block device for ceph to use. This information is currently not collected by the block devices host collector.

Describe the feature

• Collector The collector parses lsblk output using these specified columns. We can extract more information from lsblk output by adding more flags that contain partition table information which the analyser would use to evaluate if a block device is partitioned or not. Here is the output from a partitioned and non-partitioned host.

Partitioned

$ sudo lsblk /dev/sdb --noheadings --bytes --pairs -o NAME,PTUUID,PTTYPE,PARTTYPE,PARTLABEL,PARTFLAGS
NAME="sdb" PTUUID="906db4f7-f33e-43e5-98f2-2218e41ba814" PTTYPE="gpt" PARTTYPE="" PARTLABEL="" PARTFLAGS=""
NAME="sdb1" PTUUID="906db4f7-f33e-43e5-98f2-2218e41ba814" PTTYPE="gpt" PARTTYPE="0fc63daf-8483-4772-8e79-3d69d8477de4" PARTLABEL="primary" PARTFLAGS=""

Unpartitioned

$ sudo lsblk /dev/sdb --noheadings --bytes --pairs -o NAME,PTUUID,PTTYPE,PARTTYPE,PARTLABEL,PARTFLAGS
NAME="sdb" PTUUID="" PTTYPE="" PARTTYPE="" PARTLABEL="" PARTFLAGS=""

• Analyser Update the block devices analyser to expose this information i.e whether a disk type device is partitioned or not. Also consider other device types e.g raid.

Describe alternatives you've considered

Leave things as they are and not catch this use case

[St0rmz1]

Hi @banjoh Any idea if this is on the roadmap?

[banjoh]

@St0rmz1 its not yet been scheduled in the roadmap. Is there an urgent need of having the issue worked on?

[banjoh]

After some investigation, it turns out this is a non-issue. The block devices collector/analyser behaves as should. Ceph is capable of using a partition as long as it has not been formatted. Here we can see ceph (1.10) successfully using a partition block.

evans@evans-vm2:~$ lsblk /dev/sdd --json --output NAME,PKNAME,TYPE,FSTYPE,PTTYPE
{
   "blockdevices": [
      {
         "name": "sdd",
         "pkname": null,
         "type": "disk",
         "fstype": null,
         "pttype": "gpt",
         "children": [
            {
               "name": "sdd1",
               "pkname": "sdd",
               "type": "part",
               "fstype": "ceph_bluestore",
               "pttype": "gpt"
            },{
               "name": "sdd2",
               "pkname": "sdd",
               "type": "part",
               "fstype": "crypto_LUKS",
               "pttype": "gpt"
            }
         ]
      }
   ]
}

[xavpaice]

See notes in #1122 / #1121 where the earlier approach failed collection for older kernels. Looks like what we're going to need to do is detect the kernel version and collect the right columns for that version.

[banjoh]

The first attempt to collect partition information needed to be reverted cause older versions of lsblk did not have PTTYPE field and a few others.

The second attempt ought to check whether the field is supported by the version of lsblk installed in that OS. The only place I could find this information is in the output of the lsblk --help

Older versions

[evans] $ docker run -it --rm ubuntu:trusty bash -c "lsblk --help"

Usage:
 lsblk [options] [<device> ...]

Options:
 -a, --all            print all devices
 -b, --bytes          print SIZE in bytes rather than in human readable format
 -d, --nodeps         don't print slaves or holders
 -D, --discard        print discard capabilities
 -e, --exclude <list> exclude devices by major number (default: RAM disks)
 -f, --fs             output info about filesystems
 -h, --help           usage information (this)
 -i, --ascii          use ascii characters only
 -m, --perms          output info about permissions
 -l, --list           use list format ouput
 -n, --noheadings     don't print headings
 -o, --output <list>  output columns
 -P, --pairs          use key="value" output format
 -r, --raw            use raw output format
 -t, --topology       output info about topology

Available columns:
       NAME  device name
      KNAME  internal kernel device name
    MAJ:MIN  major:minor device number
     FSTYPE  filesystem type
 MOUNTPOINT  where the device is mounted
      LABEL  filesystem LABEL
       UUID  filesystem UUID
         RO  read-only device
         RM  removable device
      MODEL  device identifier
       SIZE  size of the device
      STATE  state of the device
      OWNER  user name
      GROUP  group name
       MODE  device node permissions
  ALIGNMENT  alignment offset
     MIN-IO  minimum I/O size
     OPT-IO  optimal I/O size
    PHY-SEC  physical sector size
    LOG-SEC  logical sector size
       ROTA  rotational device
      SCHED  I/O scheduler name
    RQ-SIZE  request queue size
       TYPE  device type
   DISC-ALN  discard alignment offset
  DISC-GRAN  discard granularity
   DISC-MAX  discard max bytes
  DISC-ZERO  discard zeroes data

For more information see lsblk(8).

More resent version

[evans] $ docker run -it --rm ubuntu:focal bash -c "lsblk --help"

Usage:
 lsblk [options] [<device> ...]

List information about block devices.

Options:
 -D, --discard        print discard capabilities
 -E, --dedup <column> de-duplicate output by <column>
 -I, --include <list> show only devices with specified major numbers
 -J, --json           use JSON output format
 -O, --output-all     output all columns
 -P, --pairs          use key="value" output format
 -S, --scsi           output info about SCSI devices
 -T, --tree[=<column>] use tree format output
 -a, --all            print all devices
 -b, --bytes          print SIZE in bytes rather than in human readable format
 -d, --nodeps         don't print slaves or holders
 -e, --exclude <list> exclude devices by major number (default: RAM disks)
 -f, --fs             output info about filesystems
 -i, --ascii          use ascii characters only
 -l, --list           use list format output
 -M, --merge          group parents of sub-trees (usable for RAIDs, Multi-path)
 -m, --perms          output info about permissions
 -n, --noheadings     don't print headings
 -o, --output <list>  output columns
 -p, --paths          print complete device path
 -r, --raw            use raw output format
 -s, --inverse        inverse dependencies
 -t, --topology       output info about topology
 -z, --zoned          print zone model
 -x, --sort <column>  sort output by <column>
     --sysroot <dir>  use specified directory as system root

 -h, --help           display this help
 -V, --version        display version

Available output columns:
        NAME  device name
       KNAME  internal kernel device name
        PATH  path to the device node
     MAJ:MIN  major:minor device number
     FSAVAIL  filesystem size available
      FSSIZE  filesystem size
      FSTYPE  filesystem type
      FSUSED  filesystem size used
      FSUSE%  filesystem use percentage
  MOUNTPOINT  where the device is mounted
       LABEL  filesystem LABEL
        UUID  filesystem UUID
      PTUUID  partition table identifier (usually UUID)
      PTTYPE  partition table type
    PARTTYPE  partition type UUID
   PARTLABEL  partition LABEL
    PARTUUID  partition UUID
   PARTFLAGS  partition flags
          RA  read-ahead of the device
          RO  read-only device
          RM  removable device
     HOTPLUG  removable or hotplug device (usb, pcmcia, ...)
       MODEL  device identifier
      SERIAL  disk serial number
        SIZE  size of the device
       STATE  state of the device
       OWNER  user name
       GROUP  group name
        MODE  device node permissions
   ALIGNMENT  alignment offset
      MIN-IO  minimum I/O size
      OPT-IO  optimal I/O size
     PHY-SEC  physical sector size
     LOG-SEC  logical sector size
        ROTA  rotational device
       SCHED  I/O scheduler name
     RQ-SIZE  request queue size
        TYPE  device type
    DISC-ALN  discard alignment offset
   DISC-GRAN  discard granularity
    DISC-MAX  discard max bytes
   DISC-ZERO  discard zeroes data
       WSAME  write same max bytes
         WWN  unique storage identifier
        RAND  adds randomness
      PKNAME  internal parent kernel device name
        HCTL  Host:Channel:Target:Lun for SCSI
        TRAN  device transport type
  SUBSYSTEMS  de-duplicated chain of subsystems
         REV  device revision
      VENDOR  device vendor
       ZONED  zone model

For more details see lsblk(8).