kustomization: hit 27s timeout running '/usr/bin/git submodule update --init --recursive'

[ar-qun]

Describe the bug

Hello everyone,

I am a little bit baffled by what exactly happened but around 3 hours a go I started receiving an error

Warning BuildFailed 6m12s (x146 over 3h33m) kustomize-controller (combined from similar events): kustomize build failed: accumulating resources: accumulation err='accumulating resources from '../../base/infrastructure': read /tmp/kustomization-1762639108/kubernetes/base/infrastructure: is a directory': recursed accumulation of path '/tmp/kustomization-1762639108/kubernetes/base/infrastructure': accumulating resources: accumulation err='accumulating resources from 'reloader/': read /tmp/kustomization-1762639108/kubernetes/base/infrastructure/reloader: is a directory': recursed accumulation of path '/tmp/kustomization-1762639108/kubernetes/base/infrastructure/reloader': accumulating resources: accumulation err='accumulating resources from 'https://github.com/stakater/Reloader/deployments/kubernetes': URL is a git repository': hit 27s timeout running '/usr/bin/git submodule update --init --recursive'

On this customization

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: reloader

resources:
  - namespace.yaml
  - https://github.com/stakater/Reloader/deployments/kubernetes

patches:
- target:
    group: apps
    version: v1
    kind: Deployment
    name: reloader-reloader
  patch: |-
    - op: add
      path: /spec/template/spec/containers/0/args
      value:
      - --auto-reload-all

Strangely enough if I run flux diff kustomization infrastructure --path kubernetes/overlay/infrastructure it works:

✓  Kustomization diffing...
► Deployment/reloader/reloader-reloader drifted

metadata.generation
  ± value change
    - 12
    + 13

spec.template.spec.containers.reloader-reloader.image
  ± value change
    - ghcr.io/stakater/reloader:v1.0.83
    + ghcr.io/stakater/reloader:v1.0.85

⚠️ identified at least one change, exiting with non-zero exit code

Thank you kindly in advance.

Steps to reproduce

Really didn't do anything here...

Expected behavior

Should just work...

Screenshots and recordings

No response

OS / Distro

WSL2

Flux version

flux: v2.2.3

Flux check

► checking prerequisites ✔ Kubernetes 1.27.10-gke.1055000 >=1.26.0-0 ► checking version in cluster ✔ distribution: flux-v2.2.3 ✔ bootstrapped: true ► checking controllers ✔ helm-controller: deployment ready ► ghcr.io/fluxcd/helm-controller:v0.37.4 ✔ image-automation-controller: deployment ready ► ghcr.io/fluxcd/image-automation-controller:v0.37.1 ✔ image-reflector-controller: deployment ready ► ghcr.io/fluxcd/image-reflector-controller:v0.31.2 ✔ kustomize-controller: deployment ready ► ghcr.io/fluxcd/kustomize-controller:v1.2.2 ✔ notification-controller: deployment ready ► ghcr.io/fluxcd/notification-controller:v1.2.4 ✔ source-controller: deployment ready ► ghcr.io/fluxcd/source-controller:v1.2.4 ► checking crds ✔ alerts.notification.toolkit.fluxcd.io/v1beta3 ✔ buckets.source.toolkit.fluxcd.io/v1beta2 ✔ gitrepositories.source.toolkit.fluxcd.io/v1 ✔ helmcharts.source.toolkit.fluxcd.io/v1beta2 ✔ helmreleases.helm.toolkit.fluxcd.io/v2beta2 ✔ helmrepositories.source.toolkit.fluxcd.io/v1beta2 ✔ imagepolicies.image.toolkit.fluxcd.io/v1beta2 ✔ imagerepositories.image.toolkit.fluxcd.io/v1beta2 ✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta1 ✔ kustomizations.kustomize.toolkit.fluxcd.io/v1 ✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2 ✔ providers.notification.toolkit.fluxcd.io/v1beta3 ✔ receivers.notification.toolkit.fluxcd.io/v1 ✔ all checks passed

Git provider

GitHub

Container Registry provider

N/A

Additional context

No response

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[stefanprodan]

We don't recommend using kustomize remote bases as it bypass Flux source-controller and its Git cache, in fact we advise users to disable them https://fluxcd.io/flux/faq/#should-i-be-using-kustomize-remote-bases

To deploy reloader you could use a HelmRelease or create a Flux GitRepository/Kustomization for it.

[ar-qun]

I will use Flux GitRepository/Kustomization then.

There is a bit hesitation to use kustomize.toolkit.fluxcd.io/v1 too often because I suppose it will break the chain of kustomize.config.k8s.io/v1beta1 files. For example if I have set an image update in kustomization.yaml like so

images:
- name: my-service
  newTag: "1234" # {"$imagepolicy": "dev:my-service:tag"}

Then I suppose to not apply to any kustomize.toolkit.fluxcd.io/v1 resources down the line. Does that make sense?

Not that I need that for the reloader.