cloud-provider-azure icon indicating copy to clipboard operation
cloud-provider-azure copied to clipboard

Cloud Node Manager in CrashloopBackoff

Open himanshuz2 opened this issue 3 years ago • 3 comments

What happened:

Deployed the Azure Cloud Provider helm install --repo https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/helm/repo cloud-provider-azure --generate-name --set infra.clusterName=${CLUSTER_NAME}

cloud-node-manager-windows-n4hks 0/1 CrashLoopBackOff 38 (3h8m ago)

What you expected to happen:

cloud-node-manager-windows runs without CrashLoopBackOff

How to reproduce it (as minimally and precisely as possible):

Create a capz cluster with windows node and deploy the cloud provider as per this https://capz.sigs.k8s.io/topics/cloud-provider-config.html

Anything else we need to know?:

https://github.com/kubernetes-sigs/cluster-api-provider-azure/issues/2349

Environment:

  • Kubernetes version (use kubectl version): 1.23.6
  • Cloud provider or hardware configuration: CAPZ 1.3.1 Azure
  • OS (e.g: cat /etc/os-release):Windows 2019 node
  • Kernel (e.g. uname -a): Linux CAPZ-Management 5.15.0-27-generic #28-Ubuntu SMP Thu Apr 14 04:55:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools: CAPZ 1.3.1
  • Network plugin and version (if this is a network-related bug): Calico Windows
  • Others:

himanshuz2 avatar Jun 02 '22 20:06 himanshuz2

I'm wondering if service cidr is changed, Could you please provide kubeconfig file for node manager as described here https://github.com/kubernetes-sigs/cloud-provider-azure/blob/832999275a2876ed3521e3da4f39b30ffa877717/helm/cloud-provider-azure/values.yaml#L55 ?

MartinForReal avatar Jun 20 '22 02:06 MartinForReal

I haven't preserved the cluster to reproduce this problem. I will update when I get a chance to try it again. Why would service cidr change when I deploy the cluster using CAPZ and immediately deploy the cloud provider after that. CAPZ should have added the correct kubeconfig file on the windows node at the time of deployment because windows node is shown as joined to the cluster. If capz is not adding the kubeconfig file then the cloud provider helm chart should not rely on it to get the values it needs.

himanshuz2 avatar Jun 20 '22 14:06 himanshuz2

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Sep 18 '22 15:09 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Oct 18 '22 15:10 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-triage-robot avatar Nov 17 '22 15:11 k8s-triage-robot

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Reopen this issue with /reopen
  • Mark this issue as fresh with /remove-lifecycle rotten
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar Nov 17 '22 15:11 k8s-ci-robot

Re-opening this issue. I am facing this issue with the following error: Running C:\hpc\/cloud-node-manager.exe --node-name=cxp-workl-p5szn --use-instance-metadata=true --kubeconfig=C:\hpc\/var/lib/cpaw/kubeconfig.conf invalid configuration: unable to read certificate-authority C:\hpc\var\lib\cpaw\var\run\secrets\kubernetes.io\serviceaccount\ca.crt for default due to open C:\hpc\var\lib\cpaw\var\run\secrets\kubernetes.io\serviceaccount\ca.crt: The system cannot find the path specified.

The cluster is created using CAPI+CAPZ. The cloud-node-manager on windows fails a few times before it starts working. The container restarts 3-4 times with above error then it automatically starts working.

rahulgupta999 avatar May 02 '23 15:05 rahulgupta999

/remove-lifecycle rotten

rahulgupta999 avatar May 02 '23 15:05 rahulgupta999

/reopen

rahulgupta999 avatar May 02 '23 15:05 rahulgupta999

@rahulgupta999: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot avatar May 02 '23 15:05 k8s-ci-robot