AKS icon indicating copy to clipboard operation
AKS copied to clipboard

[Question] GitOps (FluxCD), HelmRepisitory, HelmRelease - code works in one namespace but doesn't in the other

Open macpak opened this issue 2 years ago • 35 comments

Describe scenario I've enabled GitOps for a repository that contains HelmRepository/HelmRelease for instaling Istio. GitOps is enabled in flux-system namespace, Cluster wide.

In my repo, under /istio/system I've 2 files:

  • istio.yaml
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: HelmRepository
metadata:
  name: istio
  namespace: istio-system
  labels:
    app.kubernetes.io/part-of: istio
spec:
  interval: 1m0s
  url: https://istio-release.storage.googleapis.com/charts
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: istio-base
  namespace: istio-system
  labels:
    app.kubernetes.io/part-of: istio
spec:
  chart:
    spec:
      version: "1.14"
      chart: base
      sourceRef:
        kind: HelmRepository
        name: istio
        namespace: istio-system
  interval: 1m0s
  • namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: istio-system

There's also a kustomization that looks like:

apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: istio-system
  namespace: flux-system
spec:
  interval: 10m0s
  path: ./istio/system
  prune: true
  wait: true
  sourceRef:
    kind: GitRepository
    name: istio

When I create a GitOps configuration in AKS, I also create a new kustomization that points to the kustomization file above. This configuration above doesn't work, no helms are installed + the istio kustomization is stuck in running health checks with a timeout. However, when I change istio-system to flux-system namespace, everything works ok. I thought that it may be related to RBAC, but checked and it seems all ClusterRoleBindings are added.

Question What's wrong with the configuration above ? Why it does work in flux-system and doesn't in istio-system ?

macpak avatar Jul 21 '22 15:07 macpak