discovery.kubelet doesn't work on GKE Autopilot

[artem-zherdiev-ingio]

What's wrong?

Hi, while trying to migrate to Grafana Alloy as one solution, we found that Kubelet doesn't work for GKE Autopilot because of usage /pods that require nodes/proxy, which is not allowed on Autopilot.

User "system:serviceaccount:NS:SA" cannot get resource "nodes/proxy" in API group "" at the cluster scope: GKE Warden authz [denied by managed-namespaces-limitation]: cluster scoped resource "nodes/proxy" is managed and access is denied"

The only one solution which works for us now is OpenTelemetry kubeletstatsreceiver which, how I see, uses /stats/summary under the hood and then create metrics based on it.

GKE Documentation says that it's not recommended to use nodes/proxy either autopilot or standard.

If your workload uses the /pods endpoint on the insecure kubelet read-only port, you need to grant the nodes/proxy RBAC permission to access the endpoint on the secure kubelet port. nodes/proxy is a powerful permission that you can't grant in GKE Autopilot clusters and that you shouldn't grant in GKE Standard clusters. Use the Kubernetes API with a fieldSelector for the node name instead.

Do you plan to make this module work on Autopilot as well?

Steps to reproduce

1. Deploy Alloy on GKE Autopilot
2. Enable discovery.kubelet
3. See Forbidden in logs

System information

GKE Autopilot v1.29.5 ~ v1.30.3

Software version

Grafana Alloy v1.4.1 / Helm Chart 0.9.0

Configuration

discovery.kubelet "default" {
  bearer_token_file = "/var/run/secrets/kubernetes.io/serviceaccount/token"
  tls_config {
    ca_file = "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt"
  }
  url = string.format("https://%s:%d", sys.env("K8S_HOST_IP"), 10250)
  refresh_interval = "30s"
}

Logs

No response