[FluxCD Integration] Update the gitops.kubesphere.io/v1alpha1 CRD to Support FluxCD Application

[chengleqi]

/kind feature

[chengleqi]

General Design

File Change

1. API

pkg/api/gitops/v1alpha1
├── application.go (UPDATE)
├── helmtemplate.go (ADD)
├── constants.go
├── groupversion_info.go
├── groupversion_info_test.go
├── image-updater.go
├── image-updater_test.go
└── zz_generated.deepcopy.go

2. Controller

controllers/fluxcd
├── git-repository-controller.go
├── application-controller.go (ADD)
├── helmtemplate-controller.go (ADD)

---

application.go

update ApplicationSpec field like below:

// ApplicationSpec is the specification of the Application
type ApplicationSpec struct {
	Kind    Engine           `json:"kind"`
	ArgoApp *ArgoApplication `json:"argoApp,omitempty"`
	FluxApp *FluxApplication `json:"fluxApp,omitempty"`
}

// constants.go
type Engine string

const (
	ArgoCD Engine = "argocd"
	FluxCD Engine = "fluxcd"
)

and FluxApplication is like below:

type FluxApplication struct {
	Spec FluxApplicationSpec `json:"spec,omitempty"`
}

type FluxApplicationSpec struct {
	Interval    metav1.Duration            `json:"interval"`
	Source      FluxApplicationSource      `json:"source"`
	Destination FluxApplicationDestination `json:"destination"`
	Config      FluxApplicationConfig      `json:"config"`
}

I think a GitOps Application should be better to abstract into three parts [Source, Destination and Config] .

the Source part is a SourceRef link to the FluxGitRepo or FluxHelmRepo we created

type FluxApplicationSource struct {
   SourceRef CrossNamespaceObjectReference `json:"sourceRef"`
}

// use case
// sourceRef:
    // kind: HelmRepository
    // name: podinfo
    // namespace: default

the Destination part:

type FluxApplicationDestination struct {
	KubeConfig *KubeConfig `json:"kubeConfig,omitempty"`
	TargetNamespace string `json:"targetNamespace,omitempty"`
}

and the Config part:

type FluxApplicationConfig struct {
	// the following four fields are the same in both flux Kustomization and flux HelmRelease CRD
	Suspend bool `json:"suspend,omitempty"`
	Timeout *metav1.Duration `json:"timeout,omitempty"`
	DependsOn []meta.NamespacedObjectReference `json:"dependsOn,omitempty"`
	ServiceAccountName string `json:"serviceAccountName,omitempty"`

	// Two Application Type
   	// same as fluxcd spec exclude the above four fields.
	HelmRelease   HelmReleaseSpec   `json:"helmRelease,omitempty"` 
        // same as fluxcd spec exclude the above four fields.
	Kustomization KustomizationSpec `json:"kustomization,omitempty"`
}

---

helmtemplate.go

helmtemplate thing is already exist in helmcharts.source.toolkit.fluxcd.io . we can use the same design like this.

example:

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmChart
metadata:
  name: podinfo
  namespace: default
spec:
  interval: 5m0s
  chart: podinfo
  reconcileStrategy: ChartVersion
  sourceRef:
    kind: HelmRepository
    name: podinfo
  version: '5.*'

this helmchart named podinfo is persisted. Because we directly told flux source-controller to create and maintain this thing. But another way to create a helmchart thing is indirectly by creating helmrelease. the flux helm-controller will detect the charts field in the yaml file to create a ephemeral helmchart. the ephemeral helmchart will be deleted when the corresponding helmrelease is deleted.

---

helmtemplate-controller.go

If we want to reuse the helm chart. We should create a controller to make the information of a helm chart persisted and may create some CRUD operations on it. And argocd application controller also can use the information of the helmtemplate.

---

application-controller.go

the main job of application-controller is to Reconcile Flux Kustomization and Flux HelmRelease resource.

use case:

the UI will provide Three GitOps Application Type [ Helm | Kustomization | Template ]

the application-controller will first check the kind of engine whether is "fluxcd".

1. when user choose Helm Type and create a new helmrelease the helmtemplate-controller will detect the ADD Event of helmcharts.source.toolkit.fluxcd.io emited by the new created helmrelease. Then the helmtemplate-controller will create a persisted helm chart same as the helm chart that created by the flux-helm-controller. Finally the application-controller will create the normal helmrelease.
2. when user choose Template Type the application-controller will find the corresponding helmtemplate according to the name. And application-controller will use the info that helmtemplate provided to fill out the helmrelease.spec.chart field then create it. But the flux helm-controller will also create a ephemeral helmchart. It's a little bit weird.

[LinuxSuRen]

Is that possible to keep using the same version v1alpha1?

[chengleqi]

Is that possible to keep using the same version v1alpha1?

Sure.

[chengleqi]

@lxm

[lxm]

We also need to establish a releation between helmrelease and helmchart using labels or annotaions.

when user choose Helm Type and create a new helmrelease the helmtemplate-controller will detect the ADD Event of helmcharts.source.toolkit.fluxcd.io emited by the new created helmrelease. Then the helmtemplate-controller will create a persisted helm chart same as the helm chart that created by the flux-helm-controller. Finally the application-controller will create the normal helmrelease

there's a option that whether user want to save the chartinfo as a template, we need to check this.

[lxm]

In addition, is it possible for us to make such a template mechanism for kustomizate

[chengleqi]

We also need to establish a releation between helmrelease and helmchart using labels or annotaions.

when user choose Helm Type and create a new helmrelease the helmtemplate-controller will detect the ADD Event of helmcharts.source.toolkit.fluxcd.io emited by the new created helmrelease. Then the helmtemplate-controller will create a persisted helm chart same as the helm chart that created by the flux-helm-controller. Finally the application-controller will create the normal helmrelease

there's a option that whether user want to save the chartinfo as a template, we need to check this.

Yes. I agree.

[chengleqi]

Four use-cases

1. User just wants to create a HelmRelease without saving the information (spec.fluxApp.spec.config.helmRelease.chart) to a Template.

apiVersion: gitops.kubesphere.io/v1alpha1
kind: Application
metadata:
  name: chengleqi-test-application-1
  namespace: my-devops-projecthmhx2
spec:
  kind: fluxcd
  fluxApp:
    spec:
      source:
        sourceRef:
          kind: GitRepository
          name: fluxcd-github-repo
          namespace: my-devops-projecthmhx2
      config:
        helmRelease:
          chart:
            interval: 5m0s
            chart: ./helm-chart
            version: "0.1.0"
            valuesFiles:
              - ./helm-chart/values.yaml
              - ./helm-chart/aliyun-values.yaml
            reconcileStrategy: Revision
          template:
          deploy:
            - destination:
                kubeConfig:
                targetNamespace: app
              interval: 1m0s
              upgrade:
                remediation:
                  remediateLastFailure: true
                force: true
              install:
                createNamespace: true

2. User wants to create a HelmRelease and save the config to a Template.

apiVersion: gitops.kubesphere.io/v1alpha1
kind: Application
metadata:
  name: chengleqi-test-application-2
  namespace: my-devops-projecthmhx2
  labels:
    gitops.kubesphere.io/save-helmtemplate: "true"
  ...

Add the gitops.kubesphere.io/save-helmtemplate: "true" label. It's the only change compare with the first case.

3. User want to create a HelmRelease use Template that saved before.

apiVersion: gitops.kubesphere.io/v1alpha1
kind: Application
metadata:
  name: chengleqi-test-application-3
  namespace: my-devops-projecthmhx2
spec:
  kind: fluxcd
  fluxApp:
    spec:
      source:
      config:
        helmRelease:
          chart:
          # template field reference a template that created before by the name
          template: my-devops-projecthmhx2-chengleqi-test-application
          deploy:
            - destination:
                kubeConfig:
                targetNamespace: app
              interval: 1m0s
              upgrade:
                remediation:
                  remediateLastFailure: true
                force: true
              install:
                createNamespace: true

4. Multi-Cluster Applications

the spec.FluxApp.config.helmRelease.deploy is an array for multi-clusters settings.

apiVersion: gitops.kubesphere.io/v1alpha1
kind: Application
metadata:
  name: chengleqi-test-application-4
  namespace: my-devops-projecthmhx2
spec:
  kind: fluxcd
  fluxApp:
    spec:
      source:
        sourceRef:
          kind: GitRepository
          name: fluxcd-github-repo
          namespace: my-devops-projecthmhx2
      config:
        helmRelease:
          chart:
          # template field reference a template that created before by the name
          template: my-devops-projecthmhx2-chengleqi-test-application
          deploy:
            - destination:
                kubeConfig:
                  secretRef:
                    name: stage-kubeconfig
                targetNamespace: app
              valuesFrom:
                - kind: ConfigMap
                  name: stage-settings
              values: 
                clouds:
                  provider: qingcloud
              install:
                remediation:
                  retries: 3
            - destination:
                kubeConfig:
                  secretRef:
                    name: prod-kubeconfig
                targetNamespace: app
              valuesFrom:
                - kind: Secret
                  name: dbpass
              values: 
                clouds:
                  provider: aliyun
              upgrade:
                remediation:
                  remediateLastFailure: true
                force: true
              interval: 1m0s

There is a description about the Multi-Clusters Application https://github.com/kubesphere/ks-devops/issues/740#issuecomment-1185308426