Alex Nordlund

Slightly redacted set of metrics, but from the same cluster for the same container ``` kube_pod_container_resource_requests{ cluster="our-us-cluster", container="carbon-relay-ng", endpoint="http-metrics", instance="10.0.0.63:8080", job="integrations/kubernetes/kube-state-metrics", namespace="grafana-agent", node="ip-10-0-0-14.ec2.internal", pod="carbon-relay-ng-594b78f7bb-npfr5", resource="memory", service="kube-state-metrics", uid="80380296-135a-429f-ab3b-c8175bbdde58", unit="byte" } ```...

Did a bit more debugging and I think I see the issue: ``` sum by (container,node,instance,pod) (container_memory_allocation_bytes{namespace="grafana-agent", instance=~"(ip-10-0-0-78.ec2.internal|10.0.0.78:10250)"}) { container="config-reloader", instance="10.0.0.78:10250", node="10.0.0.78:10250", pod="grafana-agent-node-exporter-c7529" } Value: 1916928 { container="config-reloader", instance="ip-10-0-0-78.ec2.internal", node="ip-10-0-0-78.ec2.internal",...

I don't quite understand this part, when "captures any containers that existed for a time < a prometheus scrape interval" https://github.com/opencost/opencost/blob/develop/pkg/costmodel/costmodel.go#L384 You're not actually using the same keys as everything...

Well this is curious, we do not have the same metric relabeling or config (and we're not picking up the node labels either), but we also don't have references to...

Given that you already do kubernetes service discovery on the node, and already use `__meta_kubernetes_node_name` I don't think it'd be _entirely_ unreasonably to add `node` (with the node name) as...

(that node exists may be a detail from ServiceMonitors, but I think configuring it to be present is cleaner than the current solution)

@AjayTripathy I don't think we're able to reliably reconfigure this, but with the holidays coming up I think I can do the initial change and submit a PR The actual...

@AjayTripathy the change was suspiciously easy, but it _will_ break previous opencost/kubecost deployments if applied separately I've submitted a PR both to the chart and to this repo https://github.com/opencost/opencost/pull/2423

@AjayTripathy It's been about a month since I delved into the code so I can't say for certain, but at a glance it looks like it's going to have the...

It's because alpine uses musl and there's no official nodejs for musl. There's some more info in the fork at https://github.com/node-gradle/gradle-node-plugin/issues/51