Tool to check a cluster's status like Vespa Cloud's deploy output

[kkraune]

When deploying to Vespa Cloud, it automatically checks that nodes / services are up and have correct config, like:

[INFO]     [06:21:23]  Deployment successful.
[INFO]     [06:21:23]  Session 2715 for tenant 'mytenant1' prepared and activated.
[INFO]     [06:21:23]  ######## Details for all nodes ########
[INFO]     [06:21:23]  h5569g.dev.aws-us-east-1c.vespa-external.aws.oath.cloud: expected to be UP
[INFO]     [06:21:23]  --- platform vespa/centos-tenant:7.428.19
[INFO]     [06:21:23]  --- container-clustercontroller on port 19050 has config generation 2714, wanted is 2715
[INFO]     [06:21:23]  h5575a.dev.aws-us-east-1c.vespa-external.aws.oath.cloud: expected to be UP
[INFO]     [06:21:23]  --- platform vespa/centos-tenant:7.428.19
[INFO]     [06:21:23]  --- container on port 4080 has config generation 2714, wanted is 2715
[INFO]     [06:21:23]  h5565f.dev.aws-us-east-1c.vespa-external.aws.oath.cloud: expected to be UP
[INFO]     [06:21:23]  --- platform vespa/centos-tenant:7.428.19
[INFO]     [06:21:23]  --- searchnode on port 19107 has config generation 2714, wanted is 2715
[INFO]     [06:21:23]  --- distributor on port 19111 has config generation 2715, wanted is 2715
[INFO]     [06:21:23]  --- storagenode on port 19102 has config generation 2714, wanted is 2715
[INFO]     [06:21:23]  h5575b.dev.aws-us-east-1c.vespa-external.aws.oath.cloud: expected to be UP
[INFO]     [06:21:23]  --- platform vespa/centos-tenant:7.428.19
[INFO]     [06:21:23]  --- logserver-container on port 4080 has config generation 2714, wanted is 2715

Using Vespa.ai self-hosted, one can run vespa-config-status on each node that exits with 0 if everything is OK / prints services that are not. It is a bit cumbersome, maybe vespa-model-inspect could be enhanced with the config status feature for all nodes.

Ref https://docs.vespa.ai/en/operations/admin-procedures.html