argocd-vault-plugin
argocd-vault-plugin copied to clipboard
argoproj-labs
argocd-vault-plugin with values.yaml in other git repo
Describe the bug When I deploy an application with helm, I use the following template:
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: external-dns
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: workshop
sources:
- repoURL: 'https://charts.bitnami.com/bitnami'
chart: external-dns
targetRevision: 6.*
helm:
valueFiles:
- $values/argocd/helm-values/external-dns/values.yaml
- repoURL: 'https://gitlab.com/some/project.git'
targetRevision: HEAD
ref: values
destination:
name: in-cluster
namespace: external-dns
syncPolicy:
automated:
prune: true
selfHeal: true
allowEmpty: false
syncOptions:
- CreateNamespace=true
I use the upstream Chart and add a values file from my git repo. How can I achieve this with argocd-vault-plugin running as sidecar? I dont want to write the whole values file as extra env args to the application deployment
Expected behavior I expect the plugin to work like this:
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: external-dns
namespace: argocd
spec:
project: default
sources:
- repoURL: https://charts.bitnami.com/bitnami
chart: external-dns
targetRevision: 6.31.4
plugin:
env:
- name: HELM_ARGS
value: -f $values/helm/external-dns/values.yaml
- repoURL: [email protected]/some/project.git
targetRevision: main
ref: values
destination:
name: in-cluster
namespace: external-dns
Does anyone deploy like this? I dont want to use an umbrella chart, because then I have to hardcode the repositorys at the init stage or checking the whole upstream helm chart in my git repo.
Best, Patrick
No one who has the same issues? How do you use this plugin with values? Any recommendations?
I found a workaround where I use a chart with a subchart. So my argocd application points to a git repo where my values.yaml file is but I've also got a Chart.yaml with a dependancy chart. See https://levelup.gitconnected.com/helm-dependencies-1907facbe410
Then as per this issue I also had to modify the plugin's behaviour to include the command : helm dependency build. See https://github.com/argoproj/argo-cd/issues/13009
I found a workaround where I use a chart with a subchart. So my argocd application points to a git repo where my values.yaml file is but I've also got a Chart.yaml with a dependancy chart. See https://levelup.gitconnected.com/helm-dependencies-1907facbe410
Then as per this issue I also had to modify the plugin's behaviour to include the command : helm dependency build. See argoproj/argo-cd#13009
Can you share your Application.yaml for ArgoCD? I want to have a look how you definded the application
source:
repoURL: https://bitbucket.org/bbbbbb/my-cicd-tests.git
path: avp-tests
plugin:
name: avp-helm
In the folder avp-tests I have Chart.yaml and values.yaml
My avp plugin assumes my values file is called values.yaml otherwise I could have passed it to the plugin as an argument
And here's my plugin
avp-helm:
# Note: this command is run _before_ any Helm templating is done, therefore the logic is to check
# if this looks like a Helm chart
discover:
find:
command:
- sh
- "-c"
- "find . -name 'Chart.yaml' && find . -name 'values.yaml'"
init:
command:
- sh
- "-c"
- |
helm dependency build
generate:
# **IMPORTANT**: passing `${ARGOCD_ENV_HELM_ARGS}` effectively allows users to run arbitrary code in the Argo CD
# repo-server (or, if using a sidecar, in the plugin sidecar). Only use this when the users are completely trusted. If
# possible, determine which Helm arguments are needed by your users and explicitly pass only those arguments.
command:
- sh
- "-c"
- |
argocd-vault-plugin generate values.yaml > generated-values.yaml && helm template $ARGOCD_APP_NAME -n $ARGOCD_APP_NAMESPACE -f generated-values.yaml . --include-crds
lockRepo: false
I need this as well, I have values.yaml and multiple override files that needs to be merged and secrets from Vault injected to them... it's basically the same as @en1cc's example only with that difference that I have multiple valueFiles.
same issue here. don't know how to use and even which plugin to use.
my application file -
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cert-manager
namespace: gitops
spec:
project: default
destination:
server: https://kubernetes.default.svc
namespace: security
sources:
- repoURL: "https://charts.jetstack.io"
targetRevision: 1.14.5
chart: cert-manager
helm:
valueFiles:
- $rootDir/initial resources/cert-manager/values.yaml
- repoURL: "[email protected]:my-org/helm-values.git"
plugin:
name: argocd-vault-plugin
targetRevision: main
ref: rootDir
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
and here i have my values.yaml file for cert manager.
installCRDs: true
webhook:
hostNetwork: true
securePort: 19097
extraEnv:
- name: TEST_ARGOCD_VAULT_1
value: <path:secrets-kv2/data/argocd#test>
- name: TEST_ARGOCD_VAULT_2
value: <path:secrets-kv2/argocd#test>
and here i have some logs from my vault pod
vault@vault-server-0:/$ curl --request POST -H "Content-Type: application/json" -d '{"jwt": "'$JWT'", "role": "argocd"}' http://127.0.0.1:8200/v1/auth/kubernetes/login
{
"request_id": "7ac62316-2a60-3555-5f42-85df7c554048",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": null,
"wrap_info": null,
"warnings": null,
"auth": {
"client_token": "hvs.CAESILNzaHMbitgN1KaIsiSyjdTlHbfp8kFO39t5EhqbZsFTGh4KHGh2cy5najJValUxOEttOWtmUlpjTVI1YTlLdkg",
"accessor": "CW21WDy4kBnCXxww7DNyZqE0",
"policies": ["argocd-kv-role", "default"],
"token_policies": ["argocd-kv-role", "default"],
"metadata": {
"role": "argocd",
"service_account_name": "argocd-repo-server",
"service_account_namespace": "gitops",
"service_account_secret_name": "",
"service_account_uid": "a8942c51-6bcd-4c06-8fb7-ae0dd219c457"
},
"lease_duration": 86400,
"renewable": true,
"entity_id": "0bfa6385-9191-c895-bb36-529c4fdadff4",
"token_type": "service",
"orphan": true,
"mfa_requirement": null,
"num_uses": 0
},
"mount_type": ""
}
vault@vault-server-0:/$ vault kv get secrets-kv2/argocd
===== Secret Path =====
secrets-kv2/data/argocd
======= Metadata =======
Key Value
--- -----
created_time 2024-05-15T07:01:36.073850515Z
custom_metadata <nil>
deletion_time n/a
destroyed false
version 1
==== Data ====
Key Value
--- -----
test working
and here i have config for installing official argocd helm chart
configs:
params:
server.insecure: true
cmp:
create: true
plugins:
argocd-vault-plugin:
allowConcurrency: true
discover:
find:
command:
- sh
- "-c"
- "find . -name '*.yaml' | xargs -I {} grep \"<path\\|avp\\.kubernetes\\.io\" {} | grep ."
generate:
command:
- argocd-vault-plugin
- generate
- "."
- "-s"
- "argocd-vault-plugin-configuration"
lockRepo: false
applicationSet:
enabled: false
dex:
enabled: false
controller:
replicas: 2
repoServer:
rbac:
- verbs:
- get
- list
- watch
apiGroups:
- ""
resources:
- secrets
- configmaps
volumes:
- name: argocd-cmp-cm
configMap:
name: argocd-cmp-cm
- name: custom-tools
emptyDir: {}
initContainers:
- name: download-tools
image: registry.access.redhat.com/ubi8
env:
- name: AVP_VERSION
value: 1.16.1
command: [sh, -c]
args:
- >-
curl -L https://github.com/argoproj-labs/argocd-vault-plugin/releases/download/v$(AVP_VERSION)/argocd-vault-plugin_$(AVP_VERSION)_linux_amd64 -o argocd-vault-plugin &&
chmod +x argocd-vault-plugin &&
mv argocd-vault-plugin /custom-tools/
volumeMounts:
- mountPath: /custom-tools
name: custom-tools
extraContainers:
- name: avp
command: [/var/run/argocd/argocd-cmp-server]
image: registry.access.redhat.com/ubi8
securityContext:
runAsNonRoot: true
runAsUser: 999
volumeMounts:
- mountPath: /var/run/argocd
name: var-files
- mountPath: /home/argocd/cmp-server/plugins
name: plugins
- mountPath: /tmp
name: tmp
# Register plugins into sidecar
- mountPath: /home/argocd/cmp-server/config/plugin.yaml
subPath: argocd-vault-plugin.yaml
name: argocd-cmp-cm
# Important: Mount tools into $PATH
- name: custom-tools
subPath: argocd-vault-plugin
mountPath: /usr/local/bin/argocd-vault-plugin
notifications:
enabled: true
redis:
enabled: true
# vault config
extraObjects:
- apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: argocd-vault-plugin-configuration
data:
VAULT_ADDR: aHR0cDovL3ZhdWx0LXNlcnZlci1oZWFkbGVzcy5zZWN1cml0eQ==
AVP_AUTH_TYPE: azhz
AVP_K8S_ROLE: YXJnb2Nk
AVP_TYPE: dmF1bHQ=
here is end result of env vars on cert manager TEST_ARGOCD_VAULT_1 : path:secrets-kv2/data/argocd#test TEST_ARGOCD_VAULT_2 : path:secrets-kv2/argocd#test
there is no error in logs or repo server and plugin sidecar.
Please let me know if i am doing somethiong wrong
config 2
after trying something new with application file -
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cert-manager
namespace: gitops
spec:
project: default
destination:
server: https://kubernetes.default.svc
namespace: security
sources:
- repoURL: "https://charts.jetstack.io"
targetRevision: 1.14.5
chart: cert-manager
helm:
valueFiles:
- $rootDir/initial resources/cert-manager/values.yaml
plugin:
name: argocd-vault-plugin-helm
env:
- name: HELM_VALUES
value: |
extraEnv:
- name: TEST_ARGOCD_VAULT_3
value: <path:secrets-kv2/data/argocd#test>
- repoURL: "[email protected]:my-repo/helm-values.git"
targetRevision: main
ref: rootDir
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
finally i see some error in UI -
Failed to load target state: failed to generate manifest for source 1 of 2: rpc error: code = Unknown desc = error getting app source type: multiple application sources defined: Helm,Plugin
i guess this is still not supported
tbh I'm not sure, if that's an AVP issue: it seems that all sources containing ref: aren't handed to CMPs and setting the plugin field on an helm source throws an "multiple application sources defined: Helm,Plugin" error. Both looks like restrictions by ArgoCDs Repo server.
