argocd-vault-plugin icon indicating copy to clipboard operation
argocd-vault-plugin copied to clipboard

You may need to run `helm dependency build` to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory

Open elmehdibouzar opened this issue 2 years ago • 11 comments

Describe the bug I have created my own Helm dependency that I call in another Helm repository in the Chart.yaml. When I don't use the argocd-vault-plugin plugin (sidecar), argocd pull and execute my chart correctly. But when I use it, I get an error message

Unable to create application: application spec for abj is invalid: InvalidSpecError: Unable to generate manifests in h1: rpc error: code = Unknown desc = plugin sidecar failed. error generating manifests in cmp: rpc error: code = Unknown desc = error generating manifests: sh -c helm template $ARGOCD_APP_NAME -n $ARGOCD_APP_NAMESPACE ${ARGOCD_ENV_HELM_ARGS} . | argocd-vault-plugin generate - failed exit status 1: Error: An error occurred while checking for chart dependencies. You may need to run helm dependency build to fetch missing dependencies: found in Chart.yaml, but missing in charts/ directory: app-web Error: No manifests Usage: argocd-vault-plugin generate [flags] Flags: -c, --config-path string path to a file containing Vault configuration (YAML, JSON, envfile) to use -h, --help help for generate -s, --secret-name string name of a Kubernetes Secret in the argocd namespace containing Vault configuration data in the argocd namespace of your ArgoCD host (Only available when used in ArgoCD). The namespace can be overridden by using the format :

elmehdibouzar avatar May 10 '23 13:05 elmehdibouzar

If you look at the docs https://argocd-vault-plugin.readthedocs.io/en/stable/usage/#with-helm it is recommended to run helm dependency build in the init of your CMP.

That error is coming from Helm. When you use AVP you do lose some of the built in Helm features because of the way the CMPs work in Argo CD.

werne2j avatar May 10 '23 14:05 werne2j

yes but i m using sidecare avp-helm.yaml: | --- apiVersion: argoproj.io/v1alpha1 kind: ConfigManagementPlugin metadata: name: argocd-vault-plugin-helm spec: allowConcurrency: true discover: find: command: - sh - "-c" - "find . -name 'Chart.yaml' && find . -name 'values.yaml'" generate: command: - sh

- "-c"
helm template $ARGOCD_APP_NAME --include-crds .
        argocd-vault-plugin generate -
  lockRepo: false

the init is only for argocd-cm right ? onfigManagementPlugins: |

  • name: argocd-vault-plugin-helm init: command: [sh, -c] args: ["helm dependency build"] generate: command: ["sh", "-c"] args: ["helm template $ARGOCD_APP_NAME . --include-crds | argocd-vault-plugin generate -"]

elmehdibouzar avatar May 10 '23 14:05 elmehdibouzar

No, the init is in both. https://argo-cd.readthedocs.io/en/stable/operator-manual/config-management-plugins/#sidecar-plugin

werne2j avatar May 10 '23 14:05 werne2j

I did what you asked me, I hope I didn't make any mistake, but I still have an error my dependency is located in a private Artifactory instance. For authentication, I added it as a Helm repository in ArgoCD. However, when using the plugin, the dependency is not pulled, whereas it is pulled when the plugin is not used.

Note: this command is run before any Helm templating is done, therefore the logic is to check

if this looks like a Helm chart

discover:

find:

  command:

    - sh

    - "-c"

    - "find . -name 'Chart.yaml' && find . -name 'values.yaml'"

init:

  command:

    - sh

    - "-c"

    - "helm repo update"

    - "helm dependency build"

generate:

# **IMPORTANT**: passing `${ARGOCD_ENV_helm_args}` effectively allows users to run arbitrary code in the Argo CD 

# repo-server (or, if using a sidecar, in the plugin sidecar). Only use this when the users are completely trusted. If

# possible, determine which Helm arguments are needed by your users and explicitly pass only those arguments.

command:

  - sh

  - "-c"

  - |

    helm template $ARGOCD_APP_NAME -n $ARGOCD_APP_NAMESPACE ${ARGOCD_ENV_HELM_ARGS} . |

    argocd-vault-plugin generate -

lockRepo: false

error with helm repo update : Unable to create application: application spec for abj is invalid: InvalidSpecError: Unable to generate manifests in h1: rpc error: code = Unknown desc = Manifest generation error (cached): plugin sidecar failed. error generating manifests in cmp: rpc error: code = Unknown desc = error generating manifests: sh -c helm repo update helm dependency build failed exit status 1: Error: no repositories found. You must add one before updating

error without helm repo update in the configmap Unable to create application: application spec for abj is invalid: InvalidSpecError: Unable to generate manifests in h1: rpc error: code = Unknown desc = Manifest generation error (cached): plugin sidecar failed. error generating manifests in cmp: rpc error: code = Unknown desc = error generating manifests: sh -c helm repo update helm dependency build failed exit status 1: Error: no repositories found. You must add one before updating

thank you

elmehdibouzar avatar May 10 '23 15:05 elmehdibouzar

any update ?

elmehdibouzar avatar May 11 '23 13:05 elmehdibouzar

Your init command is off. Your running this command helm repo update helm dependency build . You have to split the commands with &&. If you are trying to add a helm repo you have to run helm repo add before the dependency update. Or you can vendor the chart. Similar issue https://github.com/argoproj-labs/argocd-vault-plugin/issues/175#issuecomment-892888255

werne2j avatar May 12 '23 00:05 werne2j

For anyone searching, you can add and build the chart in the init block, like below.

     init:
        command:
          - sh
          - "-c"
          - |
            helm repo add chartname <chart-repo>
            helm dependency build  

shazinahmed avatar May 16 '23 00:05 shazinahmed

on my side this config work: (I had to add ;)

I still don't know where @shazinahmed found the syntax: chartname chart-repo !

  avp-helm.yaml: |
    ---
    apiVersion: argoproj.io/v1alpha1
    kind: ConfigManagementPlugin
    metadata:
      name: argocd-vault-plugin-helm
    spec:
      allowConcurrency: true

      # Note: this command is run _before_ any Helm templating is done, therefore the logic is to check
      # if this looks like a Helm chart
      discover:
        find:
          command:
            - sh
            - "-c"
            - "find . -name 'Chart.yaml' && find . -name 'values.yaml'"
      init:
        command:
          - sh
          - "-c"
          - |
            helm repo add chartname chart-repo;
            helm dependency build;
      generate:
        # **IMPORTANT**: passing `${ARGOCD_ENV_HELM_ARGS}` effectively allows users to run arbitrary code in the Argo CD
        # repo-server (or, if using a sidecar, in the plugin sidecar). Only use this when the users are completely trusted. If
        # possible, determine which Helm arguments are needed by your users and explicitly pass only those arguments.
        command:
          - sh
          - "-c"
          - |
            helm template $ARGOCD_APP_NAME -n $ARGOCD_APP_NAMESPACE ${ARGOCD_ENV_HELM_ARGS} . |
            argocd-vault-plugin generate -
      lockRepo: false

oDevArc avatar Aug 10 '23 12:08 oDevArc

to be honest the "good" syntax is:

      init:
        command:
          - bash
          - "-c"
          - |
            chartname=$(helm dependency list | tr -s '[:space:]' | tail -n 1 | tr -s '[:space:]' | cut -f1);
            chartrepo=$(helm dependency list | tr -s '[:space:]' | tail -n 1 | tr -s '[:space:]' | cut -f3);
            helm repo add $chartname $chartrepo;
            helm dependency build;

oDevArc avatar Oct 10 '23 09:10 oDevArc

This init command is only working for charts with exactly one dependency though. Additional ones are ignored. If the chart has none, then this will fail, since it will try to use the header line in the command (helm repo add NAME REPOSITORY (literally)).

Also I would not recommend to use the chart name as the name of the repository. You could have multiple dependencies from the same repository. And adding the same repo multiple times with different names will fail as well.

My take on this is the following:

init:
  command:
    - /bin/sh
    - -c
    - |
      #!/usr/bin/env bash
      set -Eeuo pipefail
      
      # add all repositories from this chart
      for REPO_URL in $(helm dependency list | tail -n+2 | tr -s '[:space:]' | cut -f3)
      do
        helm repo add $(echo -n "${REPO_URL}" | base64) "${REPO_URL}"
      done
      
      # finally downloading the charts dependencies
      helm dependency build

(the trailing ; are not necessary when you execute a multiline bash script)

Using the base64 encoded URL as name for the repository makes sure, that a second execution of helm repo add for the same repo will add it with the exact same name which will not fail the command.

EDIT: Changed to echo -n in script to prevent printing of newlines in some cases

maxnitze avatar Oct 17 '23 15:10 maxnitze

I spent quite some time trying to set this plugin up for applications with dependencies. Turns out it's easier to not go through all these hoops to restore default behavior and instead use https://github.com/crumbhole/argocd-lovely-plugin which has a nice feature to chain plugins.

There's also a build of it which has this vault plugin bundled and ready to use out of the box.

PrayagS avatar Jun 06 '24 02:06 PrayagS