Kubernetes.container.cpu.usage.node.pct reports capacity rather than documented allocatable usage
According to the docs here: https://www.elastic.co/guide/en/beats/metricbeat/current/exported-fields-kubernetes.html
The kubernetes.container.cpu.usage.node.pct and kubernetes.container.memory.usage.node.pct are supposed to report the node allocatable usage percentages, however they actually report node capacity percentages.
The relevant code is here: https://github.com/elastic/beats/blob/c77302852ef913c7b4e1ed521528df9a7e8ca55d/metricbeat/module/kubernetes/util/kubernetes.go#L627
<Snip inline below?
}
switch r := r.(type) {
case *kubernetes.Pod:
return map[string]mapstr.M{id: specificMetaGen.Generate(r)}
case *kubernetes.Node:
nodeName := r.GetObjectMeta().GetName()
metrics := NewNodeMetrics()
if cpu, ok := r.Status.Capacity["cpu"]; ok {
if q, err := resource.ParseQuantity(cpu.String()); err == nil {
metrics.CoresAllocatable = NewFloat64Metric(float64(q.MilliValue()) / 1000)
}
}
if memory, ok := r.Status.Capacity["memory"]; ok {
if q, err := resource.ParseQuantity(memory.String()); err == nil {
metrics.MemoryAllocatable = NewFloat64Metric(float64(q.Value()))
}
}
nodeStore, _ := metricsRepo.AddNodeStore(nodeName)
Indeed changing the the code from "Capacity" to "Allocatable" in the line above fixes the problem. I notice the Memory usage calculation is also likewise incorrect on line 632 of the above source.
Can this be fixed please?
Thanks,
Kevin
For confirmed bugs, please report:
- Version: 8.11.1 metricbeat container image.
- Operating System: Linux
- Discuss Forum URL: https://discuss.elastic.co/t/kubernetes-container-cpu-usage-node-pct-reports-capacity-rather-than-documented-allocatable-usage/360962/1
- Steps to Reproduce: Run metricbeat container with the kubernetes cpu metricset enabled.
Hi! We just realized that we haven't looked into this issue in a while. We're sorry!
We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!
Any updates on this?