feat: Allow Matrix generator to have two Git child generators without conflict

[Lobstrosity]

Summary

Matrix generators don’t allow items from the child generators to have identical keys with distinct values. This prevents having a Matrix generator where both child generators are Git generators, since they’ll auto-generate conflicting path-related parameters.

Motivation

Ability to do something like the following, where there are no conflicting parameters:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: my-apps
spec:
  generators:
    - matrix:
        generators:
          # `apps` repo defines apps to deploy. the `basename` values from this
          # repo are app names.
          - git:
              repoURL: https://github.com/org/apps.git
              pathParamPrefix: apps
              files:
                - path: '*'
          # `targets` repo defines clusters to which to deploy each app. there's
          # a directory per app interpolated into this repo's path (using
          # functionality from PR#9080) and the `basename` values from this repo
          # are the clusters that each app should deploy to.
          - git:
              repoURL: https://github.com/org/targets.git
              pathParamPrefix: targets
              files:
                - path: '{{apps.path.basename}}/*'
template:
  metadata:
    # app- and target-specific name for each `Application`.
    name: '{{apps.path.basename}}-{{targets.path.basename}}'
  spec:
    # ...
    destination:
      # assume there's a cluster secret for each target cluster where the
      # secret has the same name as the cluster.
      server: '{{targets.path.basename}}'

Proposal

Based on this Slack thread, add an optional prefix to Git generator config (pathParamPrefix in the sample above) that will be included in all path-related parameter names.

[richardjennings]

Great idea! 👍

[crenshaw-dev]

@Lobstrosity this is still on my radar, just don't want it to be a blocker for 2.5. 2.6 should have a relatively quick turnaround.

[gaeljw]

Just a word to say it works like a charm in my setup, definitely something I was waiting for. Thanks a lot!

[boedy]

This is exactly what I needed! Many thanks! 🙏

My use-case might be useful for others as reference. This PR allows me to structure my repo in a way such that an app is only generated if there's an overlay defined for the specific environment.

This is an example app-set definition used for production:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: production-apps
  namespace: argocd
spec:
  generators:
  - matrix:
      generators:
        - git:
            repoURL: https://github.com/xxxxxxxx/xxxxxxxx
            revision: main
            pathParamPrefix: "app"
            files:
            - path: "components/**/app-config.yaml"
        - git:
            repoURL: https://github.com/xxxxxxxx/xxxxxxxx
            revision: main
            pathParamPrefix: "overlay"
            files:
            - path: '{{app.path}}/overlays/production/kustomization.yaml'
  template:
    metadata:
      name: '{{app.path.basename}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/xxxxxxxx/xxxxxxxx
        targetRevision: main
        path: '{{overlay.path}}'
      destination:
        server: https://kubernetes.default.svc

Example repo structure

apps
├── webapp-1
│   ├── app-config.yaml
│   ├── base
│   │   └── kustomization.yaml
│   └── overlays
│       ├── production
│       │   └── kustomization.yaml
│       └── staging
│           └── kustomization.yaml
└─── webapp-2
    ├── app-config.yaml
    ├── base
    │   └── kustomization.yaml
    └── overlays
        └── staging
            └── kustomization.yaml

With above example, only webapp-1 would be listed as webapp-2 has no production overlay defined.

Benefits of this folder structure

• Can use the basename of app folder as the app name
• All app specific configuration is bundled together under the app dir
• When a solution for https://github.com/argoproj/argo-cd/issues/11164 is implemented app-config.yaml can contain app specific templating variables (e.g configuring autoSync)

[mfabbri]

This is a very useful feature but why not to include the option to have a dynamic git repoURL in the child generator? Something like

spec:
  generators:
    - matrix:
        generators:
        - git: 
            repoURL: https://<gitpath>/_git/management
            revision: HEAD
            files:
            - path: config/customers/**/config.yaml
        - git:
            repoURL: "{{spec.sourceRepos[0]}}"
            revision: HEAD
            files:
            - path: stacks/**/config.json

Would it be possible? Or this feature is already part of 2.6 plan?

Thanks