bootstrap failed with health check failure

[gopisaba]

Hi, we are upgrading the provider version from 0.15.3 to 1.1.1 and fluxcd version from v0.35.0 to v2.1.1 at the same. After removing the resources as mentioned here, the flux_bootstrap_git failing apply. It fails with the error message bootstrap failed with 1 health check failure(s) but I could see all the sources and kustomizations are in ready state.

How to figure out what health check is failing?

> flux check
flux get sources all
flux get kustomizations
► checking prerequisites
✔ Kubernetes 1.26.7-eks-2d98532 >=1.25.0-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.36.1
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v1.1.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v1.1.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v1.1.1
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta2
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1
✔ helmcharts.source.toolkit.fluxcd.io/v1beta2
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta2
✔ receivers.notification.toolkit.fluxcd.io/v1
✔ all checks passed
NAME                           	REVISION              	SUSPENDED	READY	MESSAGE
gitrepository/flux-paas	        master@sha1:7a0f85c1  	False    	True 	stored artifact for revision 'master@sha1:7a0f85c1'
gitrepository/flux-system      	master@sha1:7e5fc775	  False    	True 	stored artifact for revision 'master@sha1:7e5fc775'
gitrepository/core     	        master@sha1:08bf88ce	  False    	True 	stored artifact for revision 'master@sha1:08bf88ce'

NAME                                   	REVISION       	SUSPENDED	READY	MESSAGE
helmrepository/autoscaler              	sha256:cfae13b0	False    	True 	stored artifact: revision 'sha256:cfae13b0'
helmrepository/aws-ebs-csi-driver      	sha256:b6c40ae6	False    	True 	stored artifact: revision 'sha256:b6c40ae6'
helmrepository/bitnami                 	sha256:ec2c8fd2	False    	True 	stored artifact: revision 'sha256:ec2c8fd2'
helmrepository/eks                     	sha256:30178a8d	False    	True 	stored artifact: revision 'sha256:30178a8d'
helmrepository/fluent                  	sha256:aac6af34	False    	True 	stored artifact: revision 'sha256:aac6af34'
helmrepository/jetstack                	sha256:20447a0b	False    	True 	stored artifact: revision 'sha256:20447a0b'
helmrepository/metrics-server          	sha256:063bd480	False    	True 	stored artifact: revision 'sha256:063bd480'
helmrepository/prometheus              	sha256:747ec6b8	False    	True 	stored artifact: revision 'sha256:747ec6b8'

NAME                                                   	REVISION	SUSPENDED	READY	MESSAGE
helmchart/infra-monitoring-metrics-server              	3.8.4   	False    	True 	pulled 'metrics-server' chart with version '3.8.4'
helmchart/infra-monitoring-prometheus-adapter          	4.1.1   	False    	True 	pulled 'prometheus-adapter' chart with version '4.1.1'
helmchart/infra-monitoring-prometheus-blackbox-exporter	7.1.0   	False    	True 	pulled 'prometheus-blackbox-exporter' chart with version '7.1.0'
helmchart/infra-monitoring-prometheus-operator         	47.0.0  	False    	True 	pulled 'kube-prometheus-stack' chart with version '47.0.0'
helmchart/infra-tools-cert-manager                     	v1.9.1  	False    	True 	pulled 'cert-manager' chart with version 'v1.9.1'
helmchart/kube-system-aws-ebs-csi-driver               	2.17.2  	False    	True 	pulled 'aws-ebs-csi-driver' chart with version '2.17.2'
helmchart/kube-system-fluent-bit                       	0.37.0  	False    	True 	pulled 'fluent-bit' chart with version '0.37.0'

NAME                    	REVISION              	SUSPENDED	READY	MESSAGE
alerting-rules          	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
aws-ebs-csi-driver      	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
blackbox-exporter       	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
cert-manager            	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
certificates            	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
fluent-bit              	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
flux-paas               	master@sha1:7a0f85c1  False    	True 	Applied revision: master@sha1:7a0f85c1
flux-system             	master@sha1:7e5fc775	False    	True 	Applied revision: master@sha1:7e5fc775
fluxcd-monitoring       	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
grafana-dashboards      	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
helm-repositories       	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
metrics-server          	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
core            	        master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
prometheus-adapter      	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
prometheus-misc         	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce
prometheus-operator     	master@sha1:08bf88ce	False    	True 	Applied revision: master@sha1:08bf88ce