kube-state-metrics
kube-state-metrics copied to clipboard
Published
20 hours ago •
kubernetes
kubernetes
pkg/customresourcestate implement info and stateSet metric type and refactor configuration file
What this PR does / why we need it:
- Adds detection of booleans in string format to getNum.
- Refactors configuration file to allow definition of different metric types having different configuration variables.
- Refactor order of types and funcs / file structure in pkg/customersourcestate.
I'm opening this PR to start discussing possible solutions :-) I'm open to all feedback 👍
From a user perspective, this PR basically refactors the layout of the custom resource config file.
It introduces a type variable as well as typed metric configuration for Gauge, Info and StateSet metrics, which aligns its naming from the OpenMetrics specification.
The type variable aligns with the recommendations of the sig-api-machinery regarding the discriminator field.
How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality)
Does not change cardinality.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1755
Fixes #1762
Example configuration:
kind: CustomResourceStateMetrics
spec:
resources:
- subsystem: machinedeployment
namespace: capi
groupVersionKind:
group: cluster.x-k8s.io
kind: MachineDeployment
version: v1beta1
metrics:
# a gauge per metric
- name: status_conditions
each:
type: Gauge
gauge:
path:
- status
- conditions
valueFrom:
- status
labelFromKey: reason
labelsFromPath:
type: [type]
# a gauge metric
- name: spec_replicas
each:
type: Gauge
gauge:
path:
- spec
- replicas
# a info metric
- name: info
each:
type: Info
info:
labelsFromPath:
version: [spec, template, spec, version]
# a stateSet metric
- name: phase
each:
type: StateSet
stateSet:
path:
- status
- phase
list:
- Running
- ScalingUp
labelName: phase
Example resulting metrics:
# HELP capi_machinedeployment_status_conditions
# TYPE capi_machinedeployment_status_conditions gauge
capi_machinedeployment_status_conditions{type="Available"} 1
capi_machinedeployment_status_conditions{type="Ready"} 1
# HELP capi_machinedeployment_spec_replicas
# TYPE capi_machinedeployment_spec_replicas gauge
capi_machinedeployment_spec_replicas 3
# HELP capi_machinedeployment_info
# TYPE capi_machinedeployment_info gauge
capi_machinedeployment_info{version="v1.23.3"} 1
# HELP capi_machinedeployment_phase
# TYPE capi_machinedeployment_phase gauge
capi_machinedeployment_phase{phase="Running"} 1
capi_machinedeployment_phase{phase="ScalingUp"} 0
Example object
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
annotations:
machinedeployment.clusters.x-k8s.io/revision: "1"
creationTimestamp: "2022-07-01T15:53:43Z"
generation: 1
labels:
cluster.x-k8s.io/cluster-name: capi-quickstart
topology.cluster.x-k8s.io/deployment-name: md-0
topology.cluster.x-k8s.io/owned: ""
name: capi-quickstart-md-0-ttbq9
namespace: default
ownerReferences:
- apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
name: capi-quickstart
uid: f6edeaf4-3dfd-4454-86f1-ee47b5d4ff94
resourceVersion: "372237"
uid: fb42cf9b-ecb7-4484-b81c-6d945bb2dc00
spec:
clusterName: capi-quickstart
minReadySeconds: 0
progressDeadlineSeconds: 600
replicas: 3
revisionHistoryLimit: 1
selector:
matchLabels:
cluster.x-k8s.io/cluster-name: capi-quickstart
topology.cluster.x-k8s.io/deployment-name: md-0
topology.cluster.x-k8s.io/owned: ""
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
type: RollingUpdate
template:
metadata:
labels:
cluster.x-k8s.io/cluster-name: capi-quickstart
topology.cluster.x-k8s.io/deployment-name: md-0
topology.cluster.x-k8s.io/owned: ""
spec:
bootstrap:
configRef:
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
name: capi-quickstart-md-0-bootstrap-7tv7j
namespace: default
clusterName: capi-quickstart
infrastructureRef:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: DockerMachineTemplate
name: capi-quickstart-md-0-infra-tqqbt
namespace: default
version: v1.23.3
status:
availableReplicas: 3
conditions:
- lastTransitionTime: "2022-07-04T06:44:42Z"
status: "True"
type: Ready
- lastTransitionTime: "2022-07-04T06:44:42Z"
status: "True"
type: Available
observedGeneration: 1
phase: Running
readyReplicas: 3
replicas: 3
selector: cluster.x-k8s.io/cluster-name=capi-quickstart,topology.cluster.x-k8s.io/deployment-name=md-0,topology.cluster.x-k8s.io/owned=
unavailableReplicas: 0
updatedReplicas: 3
Example config:
kind: CustomResourceStateMetrics
spec:
resources:
- groupVersionKind:
group: cluster.x-k8s.io
kind: Cluster
version: v1beta1
namespace: capi
subsystem: cluster
labelsFromPath:
name: [metadata, name]
namespace: [metadata, namespace]
uid: [metadata, uid]
metrics:
- each:
gauge:
path:
- metadata
- creationTimestamp
type: Gauge
name: created
help: Unix creation timestamp.
- each:
gauge:
nilIsZero: true
path:
- spec
- paused
type: Gauge
name: spec_paused
- each:
stateSet:
labelName: status
labelsFromPath:
type:
- type
list:
- 'True'
- 'False'
- Unknown
path:
- status
- conditions
valueFrom:
- status
type: StateSet
name: status_condition
help: The current status conditions of a cluster.
- each:
stateSet:
labelName: phase
list:
- Pending
- Provisioning
- Provisioned
- Deleting
- Failed
- Unknown
path:
- status
- phase
type: StateSet
name: status_phase
- groupVersionKind:
group: controlplane.cluster.x-k8s.io
kind: KubeadmControlPlane
version: v1beta1
namespace: capi
subsystem: kubeadmcontrolplane
labelsFromPath:
name: [metadata, name]
namespace: [metadata, namespace]
uid: [metadata, uid]
cluster_name: [metadata, ownerReferences, "[kind=Cluster]", name]
metrics:
- each:
gauge:
path:
- metadata
- creationTimestamp
type: Gauge
name: created
help: Unix creation timestamp.
- each:
stateSet:
labelName: status
labelsFromPath:
type:
- type
list:
- 'True'
- 'False'
- Unknown
path:
- status
- conditions
valueFrom:
- status
type: StateSet
name: status_condition
help: The current status conditions of a kubeadmcontrolplane.
- each:
gauge:
path:
- status
- replicas
nilIsZero: true
type: Gauge
name: status_replicas
- each:
gauge:
path:
- status
- readyReplicas
nilIsZero: true
type: Gauge
name: status_replicas_ready
- each:
gauge:
path:
- status
- unavailableReplicas
nilIsZero: true
type: Gauge
name: status_replicas_unavailable
- each:
gauge:
path:
- status
- updatedReplicas
nilIsZero: true
type: Gauge
name: status_replicas_updated
- each:
gauge:
path:
- spec
- replicas
type: Gauge
name: spec_replicas
- each:
gauge:
path:
- spec
- rolloutStrategy
- rollingUpdate
- maxSurge
type: Gauge
name: spec_strategy_rollingupdate_max_surge
- each:
info:
labelsFromPath:
owner_is_controller:
- controller
owner_kind:
- kind
owner_name:
- name
owner_uid:
- uid
path:
- metadata
- ownerReferences
type: Info
name: owner
- each:
info:
labelsFromPath:
version:
- spec
- version
type: Info
name: info
- groupVersionKind:
group: cluster.x-k8s.io
kind: Machine
version: v1beta1
namespace: capi
subsystem: machine
labelsFromPath:
name: [metadata, name]
namespace: [metadata, namespace]
uid: [metadata, uid]
cluster_name: [spec, clusterName]
metrics:
- each:
gauge:
path:
- metadata
- creationTimestamp
type: Gauge
name: created
help: Unix creation timestamp.
- each:
stateSet:
labelName: status
labelsFromPath:
type:
- type
list:
- 'True'
- 'False'
- Unknown
path:
- status
- conditions
valueFrom:
- status
type: StateSet
name: status_condition
help: The current status conditions of a machine.
- each:
stateSet:
labelName: phase
list:
- Pending
- Provisioning
- Provisioned
- Running
- Deleting
- Deleted
- Failed
- Unknown
path:
- status
- phase
type: StateSet
name: status_phase
- each:
info:
labelsFromPath:
owner_is_controller:
- controller
owner_kind:
- kind
owner_name:
- name
owner_uid:
- uid
path:
- metadata
- ownerReferences
type: Info
name: owner
- each:
info:
labelsFromPath:
node_name:
- status
- nodeRef
- name
node_uid:
- status
- nodeRef
- uid
type: Info
name: status_noderef
- each:
info:
labelsFromPath:
failure_domain:
- spec
- failureDomain
internal_ip:
- status
- addresses
- "[type=InternalIP]"
- address
provider_id:
- spec
- providerID
version:
- spec
- version
type: Info
name: info
- groupVersionKind:
group: cluster.x-k8s.io
kind: MachineDeployment
version: v1beta1
namespace: capi
subsystem: machinedeployment
labelsFromPath:
name: [metadata, name]
namespace: [metadata, namespace]
uid: [metadata, uid]
cluster_name: [spec, clusterName]
metrics:
- each:
stateSet:
labelName: phase
list:
- ScalingUp
- ScalingDown
- Running
- Failed
- Unknown
path:
- status
- phase
type: StateSet
name: status_phase
- each:
gauge:
path:
- metadata
- creationTimestamp
type: Gauge
name: created
help: Unix creation timestamp.
- each:
gauge:
nilIsZero: true
path:
- spec
- paused
type: Gauge
name: spec_paused
- each:
stateSet:
labelName: status
labelsFromPath:
type:
- type
list:
- 'True'
- 'False'
- Unknown
path:
- status
- conditions
valueFrom:
- status
type: StateSet
name: status_condition
help: The current status conditions of a machinedeployment.
- each:
gauge:
path:
- status
- replicas
nilIsZero: true
type: Gauge
name: status_replicas
- each:
gauge:
path:
- status
- availableReplicas
nilIsZero: true
type: Gauge
name: status_replicas_available
- each:
gauge:
path:
- status
- unavailableReplicas
nilIsZero: true
type: Gauge
name: status_replicas_unavailable
- each:
gauge:
path:
- status
- updatedReplicas
nilIsZero: true
type: Gauge
name: status_replicas_updated
- each:
gauge:
path:
- spec
- replicas
type: Gauge
name: spec_replicas
- each:
gauge:
path:
- spec
- strategy
- rollingUpdate
- maxUnavailable
type: Gauge
name: spec_strategy_rollingupdate_max_unavailable
- each:
gauge:
path:
- spec
- strategy
- rollingUpdate
- maxSurge
type: Gauge
name: spec_strategy_rollingupdate_max_surge
- each:
info:
labelsFromPath:
owner_is_controller:
- controller
owner_kind:
- kind
owner_name:
- name
owner_uid:
- uid
path:
- metadata
- ownerReferences
type: Info
name: owner
- groupVersionKind:
group: cluster.x-k8s.io
kind: MachineHealthCheck
version: v1beta1
namespace: capi
subsystem: machinehealthcheck
labelsFromPath:
name: [metadata, name]
namespace: [metadata, namespace]
uid: [metadata, uid]
cluster_name: [spec, clusterName]
metrics:
- each:
gauge:
path:
- metadata
- creationTimestamp
type: Gauge
name: created
help: Unix creation timestamp.
- each:
info:
labelsFromPath:
owner_is_controller:
- controller
owner_kind:
- kind
owner_name:
- name
owner_uid:
- uid
path:
- metadata
- ownerReferences
type: Info
name: owner
- each:
stateSet:
labelName: status
labelsFromPath:
type:
- type
list:
- 'True'
- 'False'
- Unknown
path:
- status
- conditions
valueFrom:
- status
type: StateSet
name: status_condition
help: The current status conditions of a machinehealthcheck.
- each:
gauge:
path:
- status
- expectedMachines
type: Gauge
name: status_expected_machines
- each:
gauge:
path:
- status
- currentHealthy
type: Gauge
name: status_current_healthy
- each:
gauge:
path:
- status
- remediationsAllowed
type: Gauge
name: status_remediations_allowed
- groupVersionKind:
group: cluster.x-k8s.io
kind: MachineSet
version: v1beta1
namespace: capi
subsystem: machineset
labelsFromPath:
name: [metadata, name]
namespace: [metadata, namespace]
uid: [metadata, uid]
cluster_name: [spec, clusterName]
metrics:
- each:
gauge:
path:
- metadata
- creationTimestamp
type: Gauge
name: created
help: Unix creation timestamp.
- each:
gauge:
path:
- status
- availableReplicas
nilIsZero: true
type: Gauge
name: status_available_replicas
- each:
stateSet:
labelName: status
labelsFromPath:
type:
- type
list:
- 'True'
- 'False'
- Unknown
path:
- status
- conditions
valueFrom:
- status
type: StateSet
name: status_condition
help: The current status conditions of a machineset.
- each:
gauge:
path:
- status
- replicas
nilIsZero: true
type: Gauge
name: status_replicas
- each:
gauge:
path:
- status
- fullyLabeledReplicas
type: Gauge
name: status_fully_labeled_replicas
- each:
gauge:
path:
- status
- readyReplicas
nilIsZero: true
type: Gauge
name: status_ready_replicas
- each:
gauge:
path:
- spec
- replicas
nilIsZero: true
type: Gauge
name: spec_replicas
- each:
info:
labelsFromPath:
owner_is_controller:
- controller
owner_kind:
- kind
owner_name:
- name
owner_uid:
- uid
path:
- metadata
- ownerReferences
type: Info
name: owner
Example resulting metrics:
# HELP capi_cluster_created Unix creation timestamp.
# TYPE capi_cluster_created gauge
capi_cluster_created{name="capi-quickstart",namespace="default",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 1.657197449e+09
# HELP capi_cluster_spec_paused
# TYPE capi_cluster_spec_paused gauge
capi_cluster_spec_paused{name="capi-quickstart",namespace="default",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
# HELP capi_cluster_status_condition The current status conditions of a cluster.
# TYPE capi_cluster_status_condition gauge
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="False",type="ControlPlaneInitialized",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="False",type="ControlPlaneReady",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="False",type="InfrastructureReady",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="False",type="Ready",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="False",type="TopologyReconciled",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="True",type="ControlPlaneInitialized",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 1
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="True",type="ControlPlaneReady",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 1
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="True",type="InfrastructureReady",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 1
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="True",type="Ready",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 1
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="True",type="TopologyReconciled",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 1
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="Unknown",type="ControlPlaneInitialized",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="Unknown",type="ControlPlaneReady",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="Unknown",type="InfrastructureReady",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="Unknown",type="Ready",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_condition{name="capi-quickstart",namespace="default",status="Unknown",type="TopologyReconciled",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
# HELP capi_cluster_status_phase
# TYPE capi_cluster_status_phase gauge
capi_cluster_status_phase{name="capi-quickstart",namespace="default",phase="Deleting",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_phase{name="capi-quickstart",namespace="default",phase="Failed",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_phase{name="capi-quickstart",namespace="default",phase="Pending",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_phase{name="capi-quickstart",namespace="default",phase="Provisioned",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 1
capi_cluster_status_phase{name="capi-quickstart",namespace="default",phase="Provisioning",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
capi_cluster_status_phase{name="capi-quickstart",namespace="default",phase="Unknown",uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1"} 0
# HELP capi_kubeadmcontrolplane_created Unix creation timestamp.
# TYPE capi_kubeadmcontrolplane_created gauge
capi_kubeadmcontrolplane_created{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1.657197449e+09
# HELP capi_kubeadmcontrolplane_status_condition The current status conditions of a kubeadmcontrolplane.
# TYPE capi_kubeadmcontrolplane_status_condition gauge
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="Available",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="CertificatesAvailable",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="ControlPlaneComponentsHealthy",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="EtcdClusterHealthy",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="MachinesCreated",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="MachinesReady",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="Ready",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="Resized",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="Available",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="CertificatesAvailable",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="ControlPlaneComponentsHealthy",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="EtcdClusterHealthy",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="MachinesCreated",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="MachinesReady",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="Ready",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="Resized",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="Available",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="CertificatesAvailable",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="ControlPlaneComponentsHealthy",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="EtcdClusterHealthy",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="MachinesCreated",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="MachinesReady",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="Ready",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
capi_kubeadmcontrolplane_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="Resized",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
# HELP capi_kubeadmcontrolplane_status_replicas
# TYPE capi_kubeadmcontrolplane_status_replicas gauge
capi_kubeadmcontrolplane_status_replicas{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 3
# HELP capi_kubeadmcontrolplane_status_replicas_ready
# TYPE capi_kubeadmcontrolplane_status_replicas_ready gauge
capi_kubeadmcontrolplane_status_replicas_ready{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 0
# HELP capi_kubeadmcontrolplane_status_replicas_unavailable
# TYPE capi_kubeadmcontrolplane_status_replicas_unavailable gauge
capi_kubeadmcontrolplane_status_replicas_unavailable{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 3
# HELP capi_kubeadmcontrolplane_status_replicas_updated
# TYPE capi_kubeadmcontrolplane_status_replicas_updated gauge
capi_kubeadmcontrolplane_status_replicas_updated{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 3
# HELP capi_kubeadmcontrolplane_spec_replicas
# TYPE capi_kubeadmcontrolplane_spec_replicas gauge
capi_kubeadmcontrolplane_spec_replicas{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 3
# HELP capi_kubeadmcontrolplane_spec_strategy_rollingupdate_max_surge
# TYPE capi_kubeadmcontrolplane_spec_strategy_rollingupdate_max_surge gauge
capi_kubeadmcontrolplane_spec_strategy_rollingupdate_max_surge{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
# HELP capi_kubeadmcontrolplane_owner
# TYPE capi_kubeadmcontrolplane_owner gauge
capi_kubeadmcontrolplane_owner{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",owner_is_controller="true",owner_kind="Cluster",owner_name="capi-quickstart",owner_uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1",uid="afeef341-29e8-4f5f-8e02-e04a023ac772"} 1
# HELP capi_kubeadmcontrolplane_info
# TYPE capi_kubeadmcontrolplane_info gauge
capi_kubeadmcontrolplane_info{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="afeef341-29e8-4f5f-8e02-e04a023ac772",version="v1.23.3"} 1
# HELP capi_machinedeployment_status_phase
# TYPE capi_machinedeployment_status_phase gauge
capi_machinedeployment_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",phase="Failed",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
capi_machinedeployment_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",phase="Running",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
capi_machinedeployment_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",phase="ScalingDown",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
capi_machinedeployment_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",phase="ScalingUp",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 1
capi_machinedeployment_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",phase="Unknown",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
# HELP capi_machinedeployment_created Unix creation timestamp.
# TYPE capi_machinedeployment_created gauge
capi_machinedeployment_created{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 1.657197449e+09
# HELP capi_machinedeployment_spec_paused
# TYPE capi_machinedeployment_spec_paused gauge
capi_machinedeployment_spec_paused{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
# HELP capi_machinedeployment_status_condition The current status conditions of a machinedeployment.
# TYPE capi_machinedeployment_status_condition gauge
capi_machinedeployment_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="False",type="Available",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 1
capi_machinedeployment_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="False",type="Ready",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 1
capi_machinedeployment_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="True",type="Available",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
capi_machinedeployment_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="True",type="Ready",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
capi_machinedeployment_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="Unknown",type="Available",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
capi_machinedeployment_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="Unknown",type="Ready",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
# HELP capi_machinedeployment_status_replicas
# TYPE capi_machinedeployment_status_replicas gauge
capi_machinedeployment_status_replicas{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 3
# HELP capi_machinedeployment_status_replicas_available
# TYPE capi_machinedeployment_status_replicas_available gauge
capi_machinedeployment_status_replicas_available{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
# HELP capi_machinedeployment_status_replicas_unavailable
# TYPE capi_machinedeployment_status_replicas_unavailable gauge
capi_machinedeployment_status_replicas_unavailable{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 3
# HELP capi_machinedeployment_status_replicas_updated
# TYPE capi_machinedeployment_status_replicas_updated gauge
capi_machinedeployment_status_replicas_updated{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 3
# HELP capi_machinedeployment_spec_replicas
# TYPE capi_machinedeployment_spec_replicas gauge
capi_machinedeployment_spec_replicas{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 3
# HELP capi_machinedeployment_spec_strategy_rollingupdate_max_unavailable
# TYPE capi_machinedeployment_spec_strategy_rollingupdate_max_unavailable gauge
capi_machinedeployment_spec_strategy_rollingupdate_max_unavailable{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 0
# HELP capi_machinedeployment_spec_strategy_rollingupdate_max_surge
# TYPE capi_machinedeployment_spec_strategy_rollingupdate_max_surge gauge
capi_machinedeployment_spec_strategy_rollingupdate_max_surge{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 1
# HELP capi_machinedeployment_owner
# TYPE capi_machinedeployment_owner gauge
capi_machinedeployment_owner{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",owner_kind="Cluster",owner_name="capi-quickstart",owner_uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1",uid="767112f2-7fff-4bbb-818c-4d052142aa2a"} 1
# HELP capi_machinehealthcheck_created Unix creation timestamp.
# TYPE capi_machinehealthcheck_created gauge
capi_machinehealthcheck_created{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 1.657199098e+09
capi_machinehealthcheck_created{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 1.657197449e+09
# HELP capi_machinehealthcheck_owner
# TYPE capi_machinehealthcheck_owner gauge
capi_machinehealthcheck_owner{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",owner_kind="Cluster",owner_name="capi-quickstart",owner_uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 1
capi_machinehealthcheck_owner{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",owner_kind="Cluster",owner_name="capi-quickstart",owner_uid="4e92f19d-aec5-4b7d-801e-dd3167b4a7d1",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 1
# HELP capi_machinehealthcheck_status_condition The current status conditions of a machinehealthcheck.
# TYPE capi_machinehealthcheck_status_condition gauge
capi_machinehealthcheck_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="False",type="RemediationAllowed",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 0
capi_machinehealthcheck_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="True",type="RemediationAllowed",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 1
capi_machinehealthcheck_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",status="Unknown",type="RemediationAllowed",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 0
capi_machinehealthcheck_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="False",type="RemediationAllowed",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 0
capi_machinehealthcheck_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="True",type="RemediationAllowed",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 1
capi_machinehealthcheck_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",status="Unknown",type="RemediationAllowed",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 0
# HELP capi_machinehealthcheck_status_expected_machines
# TYPE capi_machinehealthcheck_status_expected_machines gauge
capi_machinehealthcheck_status_expected_machines{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 3
capi_machinehealthcheck_status_expected_machines{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 3
# HELP capi_machinehealthcheck_status_current_healthy
# TYPE capi_machinehealthcheck_status_current_healthy gauge
capi_machinehealthcheck_status_current_healthy{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 3
capi_machinehealthcheck_status_current_healthy{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 3
# HELP capi_machinehealthcheck_status_remediations_allowed
# TYPE capi_machinehealthcheck_status_remediations_allowed gauge
capi_machinehealthcheck_status_remediations_allowed{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4",namespace="default",uid="1750ed28-9d01-4461-a0b9-a6627a436828"} 3
capi_machinehealthcheck_status_remediations_allowed{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn",namespace="default",uid="92fdf2c2-6af1-4ab6-b493-262677162b3b"} 3
# HELP capi_machine_created Unix creation timestamp.
# TYPE capi_machine_created gauge
capi_machine_created{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1.657197491e+09
capi_machine_created{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1.657197605e+09
capi_machine_created{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1.657197449e+09
capi_machine_created{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1.657197449e+09
capi_machine_created{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1.657197449e+09
capi_machine_created{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1.65719745e+09
# HELP capi_machine_status_condition The current status conditions of a machine.
# TYPE capi_machine_status_condition gauge
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="APIServerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="BootstrapReady",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="ControllerManagerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="EtcdMemberHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="EtcdPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="HealthCheckSucceeded",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="InfrastructureReady",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="NodeHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="Ready",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="False",type="SchedulerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="APIServerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="BootstrapReady",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="ControllerManagerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="EtcdMemberHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="EtcdPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="HealthCheckSucceeded",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="InfrastructureReady",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="NodeHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="Ready",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="True",type="SchedulerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="APIServerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="BootstrapReady",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="ControllerManagerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="EtcdMemberHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="EtcdPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="HealthCheckSucceeded",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="InfrastructureReady",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="NodeHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="Ready",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",status="Unknown",type="SchedulerPodHealthy",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="False",type="BootstrapReady",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="False",type="HealthCheckSucceeded",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="False",type="InfrastructureReady",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="False",type="NodeHealthy",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="False",type="Ready",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="True",type="BootstrapReady",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="True",type="HealthCheckSucceeded",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="True",type="InfrastructureReady",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="True",type="NodeHealthy",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="True",type="Ready",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="Unknown",type="BootstrapReady",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="Unknown",type="HealthCheckSucceeded",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="Unknown",type="InfrastructureReady",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="Unknown",type="NodeHealthy",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",status="Unknown",type="Ready",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="False",type="BootstrapReady",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="False",type="HealthCheckSucceeded",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="False",type="InfrastructureReady",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="False",type="NodeHealthy",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="False",type="Ready",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="True",type="BootstrapReady",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="True",type="HealthCheckSucceeded",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="True",type="InfrastructureReady",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="True",type="NodeHealthy",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="True",type="Ready",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="Unknown",type="BootstrapReady",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="Unknown",type="HealthCheckSucceeded",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="Unknown",type="InfrastructureReady",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="Unknown",type="NodeHealthy",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",status="Unknown",type="Ready",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="False",type="BootstrapReady",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="False",type="HealthCheckSucceeded",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="False",type="InfrastructureReady",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="False",type="NodeHealthy",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="False",type="Ready",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="True",type="BootstrapReady",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="True",type="HealthCheckSucceeded",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="True",type="InfrastructureReady",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="True",type="NodeHealthy",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="True",type="Ready",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="Unknown",type="BootstrapReady",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="Unknown",type="HealthCheckSucceeded",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="Unknown",type="InfrastructureReady",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="Unknown",type="NodeHealthy",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",status="Unknown",type="Ready",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="APIServerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="BootstrapReady",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="ControllerManagerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="EtcdMemberHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="EtcdPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="HealthCheckSucceeded",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="InfrastructureReady",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="NodeHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="Ready",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="False",type="SchedulerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="APIServerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="BootstrapReady",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="ControllerManagerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="EtcdMemberHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="EtcdPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="HealthCheckSucceeded",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="InfrastructureReady",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="NodeHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="Ready",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="True",type="SchedulerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="APIServerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="BootstrapReady",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="ControllerManagerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="EtcdMemberHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="EtcdPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="HealthCheckSucceeded",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="InfrastructureReady",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="NodeHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="Ready",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",status="Unknown",type="SchedulerPodHealthy",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="APIServerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="BootstrapReady",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="ControllerManagerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="EtcdMemberHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="EtcdPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="HealthCheckSucceeded",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="InfrastructureReady",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="NodeHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="Ready",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="False",type="SchedulerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="APIServerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="BootstrapReady",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="ControllerManagerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="EtcdMemberHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="EtcdPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="HealthCheckSucceeded",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="InfrastructureReady",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="NodeHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="Ready",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="True",type="SchedulerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="APIServerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="BootstrapReady",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="ControllerManagerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="EtcdMemberHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="EtcdPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="HealthCheckSucceeded",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="InfrastructureReady",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="NodeHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="Ready",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",status="Unknown",type="SchedulerPodHealthy",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
# HELP capi_machine_status_phase
# TYPE capi_machine_status_phase gauge
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Deleted",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Deleting",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Failed",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Pending",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Provisioned",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Provisioning",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Running",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",phase="Unknown",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Deleted",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Deleting",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Failed",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Pending",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Provisioned",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Provisioning",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Running",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",phase="Unknown",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Deleted",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Deleting",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Failed",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Pending",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Provisioned",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Provisioning",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Running",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",phase="Unknown",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Deleted",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Deleting",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Failed",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Pending",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Provisioned",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Provisioning",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Running",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",phase="Unknown",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Deleted",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Deleting",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Failed",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Pending",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Provisioned",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Provisioning",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Running",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",phase="Unknown",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Deleted",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Deleting",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Failed",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Pending",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Provisioned",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Provisioning",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Running",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_phase{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",phase="Unknown",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 0
# HELP capi_machine_owner
# TYPE capi_machine_owner gauge
capi_machine_owner{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",owner_is_controller="true",owner_kind="KubeadmControlPlane",owner_name="capi-quickstart-mv4tn",owner_uid="afeef341-29e8-4f5f-8e02-e04a023ac772",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_owner{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",owner_is_controller="true",owner_kind="KubeadmControlPlane",owner_name="capi-quickstart-mv4tn",owner_uid="afeef341-29e8-4f5f-8e02-e04a023ac772",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_owner{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",owner_is_controller="true",owner_kind="KubeadmControlPlane",owner_name="capi-quickstart-mv4tn",owner_uid="afeef341-29e8-4f5f-8e02-e04a023ac772",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_owner{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",owner_is_controller="true",owner_kind="MachineSet",owner_name="capi-quickstart-md-0-sncw4-5c9d56f55c",owner_uid="a88d9319-bc46-4006-aa3c-5c47fab45dde",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_owner{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",owner_is_controller="true",owner_kind="MachineSet",owner_name="capi-quickstart-md-0-sncw4-5c9d56f55c",owner_uid="a88d9319-bc46-4006-aa3c-5c47fab45dde",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
capi_machine_owner{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",owner_is_controller="true",owner_kind="MachineSet",owner_name="capi-quickstart-md-0-sncw4-5c9d56f55c",owner_uid="a88d9319-bc46-4006-aa3c-5c47fab45dde",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
# HELP capi_machine_status_noderef
# TYPE capi_machine_status_noderef gauge
capi_machine_status_noderef{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",node_name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",node_uid="13ea0767-ccfa-476a-925a-8e9d005b07f1",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626"} 1
capi_machine_status_noderef{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-rbzns",namespace="default",node_name="capi-quickstart-mv4tn-rbzns",node_uid="398da2df-4889-422d-8ce1-e34b15a1a470",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957"} 1
capi_machine_status_noderef{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-lcpnw",namespace="default",node_name="capi-quickstart-mv4tn-lcpnw",node_uid="e8bddf2d-99a3-4d1d-9de6-17212e6f154c",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1"} 1
capi_machine_status_noderef{cluster_name="capi-quickstart",name="capi-quickstart-mv4tn-8n92r",namespace="default",node_name="capi-quickstart-mv4tn-8n92r",node_uid="13adde51-f2ba-466c-bea0-88334fa9600a",uid="bb17d543-5e1b-43b3-b895-fd4480268720"} 1
capi_machine_status_noderef{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",node_name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",node_uid="b3efcd5f-b18a-48c2-96fc-70c5610097cf",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa"} 1
capi_machine_status_noderef{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",node_name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",node_uid="5277200e-a0cd-487f-b00b-63ee0b81d5dd",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9"} 1
# HELP capi_machine_info
# TYPE capi_machine_info gauge
capi_machine_info{cluster_name="capi-quickstart",internal_ip="172.18.0.9",name="capi-quickstart-mv4tn-lcpnw",namespace="default",provider_id="docker:////capi-quickstart-mv4tn-lcpnw",uid="7ff29b4d-4ecc-49b6-900c-0248480bfea1",version="v1.23.3"} 1
capi_machine_info{cluster_name="capi-quickstart",internal_ip="172.18.0.10",name="capi-quickstart-mv4tn-8n92r",namespace="default",provider_id="docker:////capi-quickstart-mv4tn-8n92r",uid="bb17d543-5e1b-43b3-b895-fd4480268720",version="v1.23.3"} 1
capi_machine_info{cluster_name="capi-quickstart",internal_ip="172.18.0.8",name="capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",namespace="default",provider_id="docker:////capi-quickstart-md-0-sncw4-5c9d56f55c-rmvs8",uid="d91bb185-001b-472f-8bc4-5abd739bb7fa",version="v1.23.3"} 1
capi_machine_info{cluster_name="capi-quickstart",internal_ip="172.18.0.6",name="capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",namespace="default",provider_id="docker:////capi-quickstart-md-0-sncw4-5c9d56f55c-9wdsf",uid="7e87ee06-6fd2-43c0-9554-c0575e6b19c9",version="v1.23.3"} 1
capi_machine_info{cluster_name="capi-quickstart",internal_ip="172.18.0.7",name="capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",namespace="default",provider_id="docker:////capi-quickstart-md-0-sncw4-5c9d56f55c-s8df9",uid="e6c7fb8b-9809-4b6c-affa-ee399ea49626",version="v1.23.3"} 1
capi_machine_info{cluster_name="capi-quickstart",internal_ip="172.18.0.5",name="capi-quickstart-mv4tn-rbzns",namespace="default",provider_id="docker:////capi-quickstart-mv4tn-rbzns",uid="8adf76ec-7144-4cff-a73f-fa40cdb2c957",version="v1.23.3"} 1
# HELP capi_machineset_created Unix creation timestamp.
# TYPE capi_machineset_created gauge
capi_machineset_created{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 1.657197449e+09
# HELP capi_machineset_status_available_replicas
# TYPE capi_machineset_status_available_replicas gauge
capi_machineset_status_available_replicas{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
# HELP capi_machineset_status_condition The current status conditions of a machineset.
# TYPE capi_machineset_status_condition gauge
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="False",type="MachinesCreated",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="False",type="MachinesReady",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="False",type="Ready",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 1
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="False",type="Resized",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 1
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="True",type="MachinesCreated",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 1
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="True",type="MachinesReady",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 1
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="True",type="Ready",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="True",type="Resized",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="Unknown",type="MachinesCreated",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="Unknown",type="MachinesReady",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="Unknown",type="Ready",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
capi_machineset_status_condition{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",status="Unknown",type="Resized",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
# HELP capi_machineset_status_replicas
# TYPE capi_machineset_status_replicas gauge
capi_machineset_status_replicas{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 3
# HELP capi_machineset_status_fully_labeled_replicas
# TYPE capi_machineset_status_fully_labeled_replicas gauge
capi_machineset_status_fully_labeled_replicas{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 3
# HELP capi_machineset_status_ready_replicas
# TYPE capi_machineset_status_ready_replicas gauge
capi_machineset_status_ready_replicas{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 0
# HELP capi_machineset_spec_replicas
# TYPE capi_machineset_spec_replicas gauge
capi_machineset_spec_replicas{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 3
# HELP capi_machineset_owner
# TYPE capi_machineset_owner gauge
capi_machineset_owner{cluster_name="capi-quickstart",name="capi-quickstart-md-0-sncw4-5c9d56f55c",namespace="default",owner_is_controller="true",owner_kind="MachineDeployment",owner_name="capi-quickstart-md-0-sncw4",owner_uid="767112f2-7fff-4bbb-818c-4d052142aa2a",uid="a88d9319-bc46-4006-aa3c-5c47fab45dde"} 1
I did use a cluster-api capd quickstart + added a machinehealthcheck to create the objects.
A thing to note:
This PR as of now would currently add a breaking change to the experimental feature configuration file for the custom resources.
If wanted we could also migrate the old config to a versioned configuration file and add the new layout as new version so it could leverage conversion funcs to stay compatible to the old configuration.
we are really looking forward to getting this merged and starting using kube-stat-metrics in Cluster API, any chance to bump this up in the backlog list? @fpetkovski (If you are back in the office/have some spare time)
Thanks for this contribution, I like the change and I think it takes CRD generation in the right direction.
One thing I did not understand is where the
susbsystemfield comes from in the example attached in the PR description.
Thank you for taking a look :-)
The subsystem field already exists today: https://github.com/kubernetes/kube-state-metrics/blob/master/pkg/customresourcestate/config.go#L101
// Subsystem defaults to the GroupVersionKind string, with invalid character replaced with _. If set to "_", no subsystem will be added.
// e.g., if GroupVersionKind is "myteam.io/v1/MyResource", Subsystem will be "myteam_io_v1_MyResource".
Subsystem string `yaml:"subsystem" json:"subsystem"`
It makes sense to use it to e.g. have consistent metric names while migrating to a other api verison of the CR.
I see, that maybe should not have gotten through. We should use apiVersion instead to be consistent with the kubernetes API. Since this feature is experimental, we should be able to make the change as long as we document it.
I see, that maybe should not have gotten through. We should use
apiVersioninstead to be consistent with the kubernetes API. Since this feature is experimental, we should be able to make the change as long as we document it.
Yes, should be fine to change, otherwise the config refactoring would also not be possible like this.
As it defaults not to apiVersion only, groupVersionKind may be the better match?
At the end the field allows to overwrite / replace the calculated groupVersionKind string.
Edit: we can't use groupVersionKind as this is already used to refer the actual GVK used to get/list/watch the API objects.
Alternative approaches to the current subsystem field:
Current example, results in metric capi_cluster_<metric-name> instead of capi_cluster_x_k8s_io_v1beta1_cluster_<metric-name>:
kind: CustomResourceStateMetrics
spec:
resources:
- groupVersionKind:
group: cluster.x-k8s.io
kind: Cluster
version: v1beta1
namespace: capi
subsystem: cluster # overwrites cluster_x_k8s_io_v1beta1_cluster
metrics:
...
Alternative 1: rename subsystem e.g. to apiVersion to overwrite generated metric name from GVK
kind: CustomResourceStateMetrics
spec:
resources:
- groupVersionKind:
group: cluster.x-k8s.io
kind: Cluster
version: v1beta1
namespace: capi
apiVersion: cluster # overwrites cluster_x_k8s_io_v1beta1_cluster
metrics:
...
Alternative 2: Toggle GVK defaulting via bool:
kind: CustomResourceStateMetrics
spec:
resources:
- groupVersionKind:
group: cluster.x-k8s.io
kind: Cluster
version: v1beta1
namespace: capi_cluster
disableGVKInName: true # disables cluster_x_k8s_io_v1beta1_cluster
metrics:
...
Of course there are also others :-)
One thing I am still not following, if we have a GVK, why do we need a subsystem/apiversion as well? Isn't this information already contained in the GVK?
One thing I am still not following, if we have a GVK, why do we need a subsystem/apiversion as well? Isn't this information already contained in the GVK?
The subsystem parameter is not used for get/list/watch resources from the API-Server, it is used for being able to adjust the metric name.
I'll try to make an example, let's consider we have the following config:
kind: CustomResourceStateMetrics
spec:
resources:
- groupVersionKind:
group: cluster.x-k8s.io
kind: Cluster
version: v1alpha4
namespace: capi
metrics:
...
The resulting metrics would be called: capi_cluster_x_k8s_io_v1alpha4_Cluster_<metric.name>.
Now a new api version gets introduced v1beta1. The configuration also now adapts the new GVK:
kind: CustomResourceStateMetrics
spec:
resources:
- groupVersionKind:
group: cluster.x-k8s.io
kind: Cluster
version: v1beta1
namespace: capi
metrics:
...
This results in also changing the metric names to capi_cluster_x_k8s_io_v1beta1_Cluster_<metric.name> and all users will have to also adjust their queries to the new metric name.
If instead the current subsystem field was set to foo for both configurations, the metric would always be capi_foo_<metric.name> and queries would not need to get adjusted when changing GVK to the next API Version. (normally you I would call this cluster, but I chose foo to highlight how the config adjusts the metric).
Got it, thanks for the explanation. In that case, this field seems to be related to the of the metric, and not to the kube api resource itself. From what I understand, the value can be arbitrary and user-defined. So something like metricNamePrefix might be more suitable?
I also don't think the CR namespace should end up in the metric name, but rather be exposed as a label. This way if metricNamePrefix is set, metrics will have the form foo_<metric.name>{namespace="capi", ...}.
Wdyt?
I agree, so the idea would be to have a field metricNamePrefix and remove subsystem as well as namespace.
- If
metricNamePrefixis not set, it calculates the prefix from the GVK, resulting in e.g.cluster_x_k8s_io_v1beta1_Cluster_<metric.name>(forapiVersion: cluster.x-k8s.io/v1beta1,kind: Cluster) - if
metricNamePrefixis set e.g. tofoo_barit would result infoo_bar_<metric.name>
👍 because namespace is also misleading because it has nothing to do with a kubernetes namespace of a custom resource
I will adjust the PR to do so :-)
On the prefix/name:
Every metric ksm exposes used kube_, does it make sense to prefix everything with kube_crd_ by default?
On the prefix/name: Every metric ksm exposes used
kube_, does it make sense to prefix everything with kube_crd_ by default?
I would prefer to at least not enforce it. When using MetricNamePrefix, kube_ could be added manually. An option may be to use kube_<GVK> when no prefix is set.
Let's definitely default to something which is consistent with other kube-state-metrics metrics.
Changed the defaulting to kube_<GVK> now, do we have consensus on that topic? If no, should we move the namespace/subsystem/metricsNamePrefix discussion to a new/separate issue?
Also did some tests and currently works great for my configuration :-)
/hold cancel
Can you update https://github.com/kubernetes/kube-state-metrics/blob/master/docs/customresourcestate-metrics.md as well?
Can you update https://github.com/kubernetes/kube-state-metrics/blob/master/docs/customresourcestate-metrics.md as well?
Yes sure 👍 thanks for pointing me to it 😃
I also updated the docs now. Happy to get another round of feedback :-)
Thanks to everyone involved 👍
Can you fix linting?
pkg/customresourcestate/registry_factory_test.go:26: File is not `goimports`-ed with -local k8s.io/kube-state-metrics (goimports)
"k8s.io/kube-state-metrics/v2/pkg/metric"
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: chrischdi, fpetkovski, mrueg
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Needs approval from an approver in each of these files:
- ~~OWNERS~~ [fpetkovski,mrueg]
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
@chrischdi Great work, thank you very much!
Looking forward to using this feature :)