[RFE] Git File Manifest Template Generator

[williamcaban]

Note: Opening in the ApplicationSet repo (original https://github.com/argoproj/argo-cd/issues/7447)

Summary

The current Git File Generator generates Applications template parameters using the contents of JSON files found within a specified repository.

This request for enhnacement (RFE) is to extend the Git File Generator (or create a new generator) to allow for the use of {{param}}-style parameters on source manifest from a Git repo and render them in-flight without requiring source modification. This is NOT to have a full templating system or DDL, it is for a simple variable substitution capability.

Proposal

For example, consider the ability to support the following scenario:

# Template with deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: '{{values.basename}}'
  namespace: '{{values.namespace}}'
  labels:
    app: '{{values.basename}}'
spec:
  replicas: '{{values.replicas}}'
  selector:
    matchLabels:
      app: '{{values.basename}}'
  template:
    metadata:
      labels:
        app: '{{values.basename}}'
    spec:
      containers:
      - name: nginx
        image: '{{values.container.image}}'
        ports:
        - containerPort: '{{values.container.port}}'

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: apps-from-template
spec:
  generators:
  - git:
      repoURL: https://repo/with/templates.git
      revision: HEAD
      templates:
        # eg: *, *.yaml, *.tmpl
        files: 'path/to/apps-from-template/deployment.yaml'
  template:
    metadata:
      name: '{{files.file-name}}'
    spec:
      project: default
      source:
        repoURL: git://git/with/templates-values.git
        targetRevision: HEAD
        # eg: *, *.json, values.json
        values: 'path/to/values-*.json'
      destination:
        server: '{{values.cluster}}'
        # Action: render `templates.files` with `source.values` to `destination.server`

With a values-mytest.json like

{
    'basename':'mytest',
    'namespace':'mynamespace',
    'replicas':'3',
    'container': {
        'image':'path/to/image:abc',
        'port':'9090'
    }
}

Note: If full templating is considered then integrating with Jinja2 or go-template would be preferred.

[RandyLevensalor]

This would be very valuable for us. We are trying to delpy several Kubernetes clusters on bare-metal. Being able to use templates directly in ArgoCD would simplify the process and make it a more complete GitOps model. Plus this could help reduce code and configuration duplication.

Thanks for suggesting this @williamcaban

[pdrastil]

I believe this might release to https://github.com/argoproj-labs/applicationset/issues/369 where we would like to use Git generator to fetch external yaml configuration from Git and put it into helm.values as a yaml blob (similar to Helm {{ toYaml .values }})

[williamcaban]

The idea with this RFE is NOT to have to use Helm and instead have it natively as a capability of applicationsets.

[ErikLundJensen]

Looking into the code I see that it is possible to address elements in an array or map.

version: 0.0.1
dependencies:
  - name: common-stuff
    version: 1.0.0

then

chartDependencies:
  version: {{dependencies.0.version}}
  name: {{dependencies.0.name}}

however, in @williamcodes particular case a 'wildcard' implementation might be a future solution. For example:

chartDependencies:
  {{dependencies.*}}