HCCM Controller don't set metadata to the Nodes at all (nodes are connected, using internal network)

[DmitrySkrotski]

TL;DR

I set up cluster on hcloud (controlplanes are cloud, workers are robots + cloud) and I am expecting that the HCCM will populate node objects with metadata.

Expected behavior

HCCM will populate information about the server to the Node object as described here hccm

Observed behavior

Nodes joined using join.yml file

kind: JoinConfiguration
discovery:
  bootstrapToken:
    apiServerEndpoint: 10.195.4.250:6443
    caCertHashes:
    - sha256:xxxxxxxxx
    token: xxxxxxx.xxxxxxxxxxxxxx
nodeRegistration:
  name: prod-k8s-metall01
  criSocket: unix:///var/run/containerd/containerd.sock
  kubeletExtraArgs:
    node-ip: 10.195.10.2

HCCM is setup using helm

helm install hcloud-controller-manager hcloud/hcloud-cloud-controller-manager --set robot.enabled=true --set netwo
rk.enabled=false -n kube-system

secret hcloud persists HCCM logs: pod initialization log:

Minimal working example

No response

Log output

plain text
I0731 08:29:07.120000       1 controllermanager.go:169] Version: v0.0.0-master+$Format:%H$
I0731 08:29:07.139351       1 secure_serving.go:213] Serving securely on [::]:10258
I0731 08:29:07.139953       1 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0731 08:29:07.140307       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
I0731 08:29:07.140913       1 shared_informer.go:313] Waiting for caches to sync for RequestHeaderAuthRequestController
I0731 08:29:07.140361       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file"
I0731 08:29:07.141232       1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0731 08:29:07.140387       1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
I0731 08:29:07.142642       1 shared_informer.go:313] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0731 08:29:07.172363       1 controllermanager.go:338] Started "cloud-node-controller"
I0731 08:29:07.172570       1 node_controller.go:164] Sending events to api server.
I0731 08:29:07.173069       1 node_controller.go:173] Waiting for informer caches to sync
I0731 08:29:07.183862       1 controllermanager.go:338] Started "cloud-node-lifecycle-controller"
I0731 08:29:07.185314       1 controllermanager.go:338] Started "service-lb-controller"
W0731 08:29:07.185470       1 core.go:111] --configure-cloud-routes is set, but cloud provider does not support routes. Will not configure cloud provider routes.
W0731 08:29:07.185532       1 controllermanager.go:326] Skipping "node-route-controller"
I0731 08:29:07.187368       1 node_lifecycle_controller.go:113] Sending events to api server
I0731 08:29:07.189552       1 controller.go:231] Starting service controller
I0731 08:29:07.189768       1 shared_informer.go:313] Waiting for caches to sync for service
I0731 08:29:07.242063       1 shared_informer.go:320] Caches are synced for RequestHeaderAuthRequestController
I0731 08:29:07.242666       1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
I0731 08:29:07.243406       1 shared_informer.go:320] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
I0731 08:29:07.290553       1 shared_informer.go:320] Caches are synced for service
E0731 08:29:07.446872       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-controlplane01": failed to get node address from cloud provider that matches ip: 10.195.4.1
E0731 08:29:07.706931       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-metall01": failed to get node address from cloud provider that matches ip: 10.195.10.2
E0731 08:29:07.831326       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-metall04": failed to get node address from cloud provider that matches ip: 10.195.10.5
E0731 08:29:08.638471       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-controlplane03": failed to get node address from cloud provider that matches ip: 10.195.4.3
E0731 08:29:08.721808       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-metall02": failed to get node address from cloud provider that matches ip: 10.195.10.3
E0731 08:29:08.816775       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-metall03": failed to get node address from cloud provider that matches ip: 10.195.10.4
E0731 08:29:08.922346       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-metall05": failed to get node address from cloud provider that matches ip: 10.195.10.6
E0731 08:29:09.079807       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-rabbit01": failed to get node address from cloud provider that matches ip: 10.195.4.4
E0731 08:29:09.261077       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-rabbit02": failed to get node address from cloud provider that matches ip: 10.195.4.5
E0731 08:34:10.240322       1 node_controller.go:389] Failed to update node addresses for node "prod-k8s-rabbit02": failed to get node address from cloud provider that matches ip: 10.195.4.5

Additional information

No response