flux2
flux2 copied to clipboard
upgrading to v0.31.3 results in a 404
Describe the bug
today on one of my clusters (EKS) I've seen this during an upgrade from v0.31.1
:
✔ cloned repository
► generating component manifests
✔ generated component manifests
✔ component manifests are up to date
✔ reconciled components
► determining if source secret "flux-system/flux-system" exists
✔ source secret up to date
► generating sync manifests
✔ generated sync manifests
✔ sync manifests are up to date
► applying sync manifests
✔ reconciled sync configuration
◎ waiting for Kustomization "flux-system/flux-system" to be reconciled
✗ failed to download artifact from http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/e36b5ede775cff77408cdc3b51091ddc3d666dd0.tar.gz, status: 404 Not Found
► confirming components are healthy
✔ helm-controller: deployment ready
✔ image-automation-controller: deployment ready
✔ image-reflector-controller: deployment ready
✔ kustomize-controller: deployment ready
✔ notification-controller: deployment ready
✔ source-controller: deployment ready
✔ all components are healthy
✗ bootstrap failed with 1 health check failure(s)
not sure if it does any harm but maybe it's worth reporting, it keeps doing that on further attempts
Steps to reproduce
after upgrading flux locally:
flux bootstrap github --context=mycontext --owner=myowner --repository=cluster --path=clusters/mycluster --branch=main --read-write-key --components-extra=image-reflector-controller,image-automation-controller
Expected behavior
no failed health checks
Screenshots and recordings
No response
OS / Distro
macOS 12.4
Flux version
v0.31.3
Flux check
► checking prerequisites
✔ Kubernetes 1.22.9-eks-a64ea69 >=1.20.6-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.22.1
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.23.4
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.19.2
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.26.2
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.24.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.25.9
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta1
✔ buckets.source.toolkit.fluxcd.io/v1beta2
✔ gitrepositories.source.toolkit.fluxcd.io/v1beta2
✔ helmcharts.source.toolkit.fluxcd.io/v1beta2
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta1
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta1
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta1
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta1
✔ receivers.notification.toolkit.fluxcd.io/v1beta1
✔ all checks passed
Git provider
GitHub
Container Registry provider
No response
Additional context
No response
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
I have the same issue on an upgrade
kustomization/core.flux-system
failed to download artifact from http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/985578b4b6083839811d59477e92d40d83654db2.tar.gz, status: 404 Not Found
revision
master/985578b4b6083839811d59477e92d40d83654db2
Can you please post here flux get source git flux-system
and flux get ks flux-system
Hi, if this is still happening, can you also share the status of the GitRepository object with kubectl -n flux-system get gitrepo flux-system -o yaml
. The status can provide more information about the state of the source artifact, if there's any issue in that.
I tried again today and got no errors, anyway:
kain@kain-osx cluster % flux get source git flux-system
NAME REVISION SUSPENDED READY MESSAGE
flux-system main/baff7f0 False True stored artifact for revision 'main/baff7f03dcfdd6635be4de0e9aedb272aee8209f'
kain@kain-osx cluster % flux get ks flux-system
NAME REVISION SUSPENDED READY MESSAGE
flux-system main/baff7f0 False True Applied revision: main/baff7f0
kain@kain-osx cluster % kubectl -n flux-system get gitrepo flux-system -o yaml
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
annotations:
reconcile.fluxcd.io/requestedAt: 2022-06-30 21:21:12.017358735 +0000 UTC m=+1287229.872381964
creationTimestamp: "2022-05-14T23:17:22Z"
finalizers:
- finalizers.fluxcd.io
generation: 1
labels:
kustomize.toolkit.fluxcd.io/name: flux-system
kustomize.toolkit.fluxcd.io/namespace: flux-system
name: flux-system
namespace: flux-system
resourceVersion: "37038373"
uid: 2e7d7c92-a9a0-42ef-8104-ea5d9c3bb9c7
spec:
gitImplementation: go-git
interval: 1m0s
ref:
branch: main
secretRef:
name: flux-system
timeout: 60s
url: ssh://[email protected]/xxx/xxx
status:
artifact:
checksum: 5035720dafc4325397bb733eb411673fc0a470ba4f6b88da468769ce0adda748
lastUpdateTime: "2022-06-30T21:21:14Z"
path: gitrepository/flux-system/flux-system/baff7f03dcfdd6635be4de0e9aedb272aee8209f.tar.gz
revision: main/baff7f03dcfdd6635be4de0e9aedb272aee8209f
size: 944967
url: http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/baff7f03dcfdd6635be4de0e9aedb272aee8209f.tar.gz
conditions:
- lastTransitionTime: "2022-06-30T21:21:14Z"
message: stored artifact for revision 'main/baff7f03dcfdd6635be4de0e9aedb272aee8209f'
observedGeneration: 1
reason: Succeeded
status: "True"
type: Ready
- lastTransitionTime: "2022-06-30T21:21:14Z"
message: stored artifact for revision 'main/baff7f03dcfdd6635be4de0e9aedb272aee8209f'
observedGeneration: 1
reason: Succeeded
status: "True"
type: ArtifactInStorage
contentConfigChecksum: sha256:fcbcf165908dd18a9e49f7ff27810176db8e9f63b4352213741664245224f8aa
lastHandledReconcileAt: 2022-06-30 21:21:12.017358735 +0000 UTC m=+1287229.872381964
observedGeneration: 1
url: http://source-controller.flux-system.svc.cluster.local./gitrepository/flux-system/flux-system/latest.tar.gz
Here's my output
➜ home-ops git:(master) flux get source git flux-system
NAME REVISION SUSPENDED READY MESSAGE
flux-system master/0fc8d21 False True stored artifact for revision 'master/0fc8d21831f141228440ab1949edf60ba3120001'
➜ home-ops git:(master) flux get ks flux-system
NAME REVISION SUSPENDED READY MESSAGE
flux-system master/0fc8d21 False True Applied revision: master/0fc8d21
The error stopped around 6/30/2022 10:30 CST
This was fixed in Flux 0.32.0