roboll
Cannot use a helm-git chart that depends on a helm-git chart
I have written two helm charts. The first helm chart lists the second helm chart in the requirements.yaml file. I don't have a helm repository, so I use helm-git to download and install these charts. This means that I have a "repository" listing for each of the charts.
When I run helmfile install, I run into trouble when helm diff tries to download the first chart:
Adding repo nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora:
Adding repo onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora: "onlyoffice-documentserver" has been added to your repositories
"onlyoffice-documentserver" has been added to your repositories
err: helm exited with status 1:
Error: no repository definition for @onlyoffice-documentserver. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: Looks like "git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora" is not a valid chart repository or cannot be reached: plugin "helm-git" exited with error
changing working directory back to "/root"
in /oas/source/repos/helmfiles/helmfile.d/20-nextcloud.yaml: helm exited with status 1:
Error: no repository definition for @onlyoffice-documentserver. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: Looks like "git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora" is not a valid chart repository or cannot be reached: plugin "helm-git" exited with error
I understand that this is all very vague, but I wanted to give a short summary of the problem before I went into specifics.
I am talking about the two charts in this repository and branch:
https://code.greenhost.net/openappstack/nextcloud/tree/4-use-onlyoffice-instead-of-collabora
(in time of writing HEAD is at this commit)
The nextcloud-onlyoffice chart includes a job that installs an integration of NextCloud and OnlyOffice, so it's requirements.yaml file lists:
dependencies:
- name: nextcloud
version: 1.1.1
repository: "@stable"
tags:
- nextcloud
- name: onlyoffice-documentserver
version: 0.1.0
repository: "@onlyoffice-documentserver"
tags:
- onlyoffice
- nextcloud
The first dependency is no problem, because it uses the @stable repository, but the second (a link to the onlyoffice-documentserver chart in the same git repository) needs you to have that helm repository available.
In the non-helmfile scenario, a user would run
helm repo add onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
To add that repository. In this case, however, we want to use helmfile. So we have this second repository with our helmfile configuration:
https://code.greenhost.net/openappstack/helmfiles/tree/7077c08573bac6b87601d6e1d7b8abd7e16ffae3
Here, in helmfile.d/20-nextcloud.yaml, we have listed the two repositories we need:
repositories:
- name: nextcloud-onlyoffice
url: git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora
- name: onlyoffice-documentserver
url: git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
But when we run the following command:
/usr/local/bin/helmfile -b /snap/bin/helm -e oas -f /oas/source/repos/helmfiles/helmfile.d/ --log-level debug apply --suppress-secrets
Helmfile exits with this error:
Adding repo nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora:
Adding repo onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
exec: /snap/bin/helm repo add onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora: "onlyoffice-documentserver" has been added to your repositories
"onlyoffice-documentserver" has been added to your repositories
err: helm exited with status 1:
Error: no repository definition for @onlyoffice-documentserver. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: Looks like "git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora" is not a valid chart repository or cannot be reached: plugin "helm-git" exited with error
changing working directory back to "/root"
in /oas/source/repos/helmfiles/helmfile.d/20-nextcloud.yaml: helm exited with status 1:
Error: no repository definition for @onlyoffice-documentserver. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: Looks like "git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora" is not a valid chart repository or cannot be reached: plugin "helm-git" exited with error
I have done some debugging, and it turns out that at some point helm-git is run with the following environment variables set:
HELM_PLUGIN_DIR=/root/.helm/plugins/helm-git
HELM_PLUGIN_NAME=helm-git
SNAP_USER_COMMON=/root/snap/helm/common
HELM_PATH_STARTER=starters
HELM_PATH_REPOSITORY=repository
HELM_BIN=/root/.helm/plugins/helm-diff/bin/diff
HELM_PLUGIN=/root/.helm/plugins
PWD=/tmp/helm-git.Ii2YPX
HELM_PATH_REPOSITORY_FILE=repository/repositories.yaml
SNAP_INSTANCE_NAME=helm
SNAP_USER_DATA=/root/snap/helm/106
HELM_PATH_CACHE=repository/cache
SNAP=/snap/helm/106
SNAP_COMMON=/var/snap/helm/common
HELM_HOME=
SNAP_DATA=/var/snap/helm/106
XDG_RUNTIME_DIR=/run/user/0/snap.helm
HELM_PATH_LOCAL_REPOSITORY=repository/local
SNAP_NAME=helm
Now I'm not sure where this is going wrong, because I'm not sure if these environment variables are set by helm-diff or by helmfile, but the problem is that HELM_HOME is set to an empty string, which results in helm-git using a temporary home directory instead of the user's normal helm home, which is the reason why it can not find the @onlyoffice-documentserver repository definition.
So I'm aware that this might be a problem in one of the plugins, but I'm not experienced enough with the whole toolset to find the exact cause of this problem on my own. Since helmfile supports using helm-git I thought I'd start here, and I'll open an issue with either plugin if we figure out that they're the culprit.
Any help would be appreciated!
@mrtndwrd Hey!
First of, do you set HELM_HOME to a custom location yourself, or just leaving it unset when running helmfile?
Also - I may be confused but would you mind removing the @ within your repo name in requirements.yaml? Wondering if it makes any difference.
Lastly, it might not be your case, but when you encounter https://github.com/aslafy-z/helm-git/issues/16 try adding a sparse=0 query to the repo url.
helm seems to empty HELM_HOME when calling a helm plugin, but that's the problem other plugins should be affected as well. That's why I'm doubting @ in the repo name is considered as an actual part of the chart name, which doesn't exist in your repo list.
Hey @mumoshu thanks for the help! It's much appreciated :)
First of, do you set HELM_HOME to a custom location yourself, or just leaving it unset when running helmfile?
I didn't, but I'll try it now to set it and see if that changes behaviour.
...
Nope. Seems like you're right: if I override HELM_HOME helm will empty it.
Also - I may be confused but would you mind removing the @ within your repo name in requirements.yaml? Wondering if it makes any difference.
If I remove the @ from the repo name in the requirements.yaml it doesn't work anymore in the normal way:
$ helm repo add onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
"onlyoffice-documentserver" has been added to your repositories
$ cd nextcloud/nextcloud-onlyoffice
$ helm dep update
Error: no repository definition for onlyoffice-documentserver. Please add them via 'helm repo add'
Note that repositories must be URLs or aliases. For example, to refer to the stable
repository, use "https://kubernetes-charts.storage.googleapis.com/" or "@stable" instead of
"stable". Don't forget to add the repo, too ('helm repo add').
I can, however, add the repo as a URL instead of an alias:
- name: onlyoffice-documentserver
version: 0.1.0
repository: "git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora"
But now when I try to install, I get this:
err: helm exited with status 1:
Error: no repository definition for git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: Looks like "git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora" is not a valid chart repository or cannot be reached: plugin "helm-git" exited with error
changing working directory back to "/root"
in /oas/source/repos/helmfiles/helmfile.d/20-nextcloud.yaml: helm exited with status 1:
Error: no repository definition for git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: Looks like "git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora" is not a valid chart repository or cannot be reached: plugin "helm-git" exited with error
So basically the same error, because however you specify the repo, you still need to have it listed when you run helm repo list, but with HELM_HOME overridden for plugin execution that doesn't work. For completeness, when I run helm repo list normally, I get this result:
$ helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
jetstack https://charts.jetstack.io
onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora
Lastly, it might not be your case, but when you encounter aslafy-z/helm-git#16 try adding a sparse=0 query to the repo url.
I tried that, but it doesn't make a difference:
...
err: release "oas-test-files" in "20-nextcloud.yaml" failed: failed processing release oas-test-files: helm exited with status 1:
Error: no repository definition for @onlyoffice-documentserver. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: plugin "helm-git" exited with error
Error: plugin "diff" exited with error
changing working directory back to "/root"
in /oas/source/repos/helmfiles/helmfile.d/20-nextcloud.yaml: failed processing release oas-test-files: helm exited with status 1:
Error: no repository definition for @onlyoffice-documentserver. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: plugin "helm-git" exited with error
Error: plugin "diff" exited with error
$ helm repo list
NAME URL
stable https://kubernetes-charts.storage.googleapis.com
local http://127.0.0.1:8879/charts
jetstack https://charts.jetstack.io
onlyoffice-documentserver git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora&sparse=0
nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora
helm seems to empty HELM_HOME when calling a helm plugin
As far as I can tell it seems to not empty it but set it to whatever is in settings.Home, but I don't really understand why that doesn't default to /root/.helm in my case :thinking:
Hi @mrtndwrd! Thanks for reporting, I'm unable to test now as it seems your git server is blocked at my workplace. Gonna try to debug when I'm back home. I added a Troubleshooting section in the helm-git repo, feel free to have a try.
Thanks @mumoshu for your input!
I started to think if you've incorrect a repository url. A repository URL must point to the parent directory of the charts, not the chart directory itself.
This is not correct:
git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collabora
But this is:
git+https://code.greenhost.net/openappstack/nextcloud?ref=4-use-onlyoffice-instead-of-collabora
Now see your helmfile.yaml and if you still have the incorrect URL under repositories, try something like:
repositories:
- name: nextcloud
url: git+https://code.greenhost.net/openappstack/nextcloud?ref=4-use-onlyoffice-instead-of-collabora
releases:
- name: nextcloud-onlyoffice
chart: nextcloud/nextcloud-onlyoffice
I can't access it via git+https as I have no read permission to it, but worth trying!
@mumoshu Sadly, helm-git still needs a path section within the repo URL, even if it's empty (see https://github.com/aslafy-z/helm-git/issues/15). So it would become:
git+https://code.greenhost.net/openappstack/nextcloud@?ref=4-use-onlyoffice-instead-of-collabora
but from what I've seen (cached google page of https://code.greenhost.net/openappstack/nextcloud), the chart is in a nextcloud subfolder (according to the google cached page), it's also possible to use:
git+https://code.greenhost.net/openappstack/nextcloud@nextcloud?ref=4-use-onlyoffice-instead-of-collabora
I understand the URL syntax could be misleading, sorry for that. It can still be discussed, I'd be pleased to have your feedback!
Thanks for your support @aslafy-z!
From what I see he's using this branch https://code.greenhost.net/openappstack/nextcloud/tree/4-use-onlyoffice-instead-of-collabora for testing, and nextcloud seems like the repository name, where there are two charts(=directories) in it:
- nextcloud-onlyoffice
- onlyoffice-documentserver
So I bet your first suggestion git+https://code.greenhost.net/openappstack/nextcloud@?ref=4-use-onlyoffice-instead-of-collabora works.
Now hoping either of the two options works :) Thansk again for your help.
Thanks for all your quick replies! That's spot on, @mumoshu
Now when I try to use that URL for the repo, I run into a chicken-and-egg problem...
I updated my requirements.yaml to this:
dependencies:
- name: nextcloud
version: 1.1.1
repository: "@stable"
tags:
- nextcloud
- name: onlyoffice-documentserver
version: 0.1.0
repository: "@nextcloud"
tags:
- onlyoffice
- nextcloud
Because the onlyoffice-documentserver chart would now be in the same repo as the nextcloud-onlyoffice chart, correct?
So then I try to add the repo as follows:
$ helm repo add nextcloud git+https://code.greenhost.net/openappstack/nextcloud@?ref=4-use-onlyoffice-instead-of-collabora
But I get this error:
Error: no repository definition for @nextcloud. Please add them via 'helm repo add'
Error in plugin 'helm-git': Error while helm_dependency_update
Error: Looks like "git+https://code.greenhost.net/openappstack/nextcloud@?ref=4-use-onlyoffice-instead-of-collabora" is not a valid chart repository or cannot be reached: plugin "helm-git" exited with error
Oh so do you mean that it would work in both cases,
@nextcloudand@?
No it shouldn't with @nextcloud, I was referring to the wrong branch (master, where there is a nextcloud folder). It could have worked with only @ but yeah, chicken and egg problem as one chart refer to the other :(
You can try that:
helm repo add nextcloud-onlyoffice git+https://code.greenhost.net/openappstack/nextcloud@nextcloud-onlyoffice?ref=4-use-onlyoffice-instead-of-collaborahelm repo add @nextcloud git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collaboraAnd then install this release:nextcloud-onlyoffice/nextcloud-onlyoffice(is this what you already tried?)
You could also try to set repository key from your dep to git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora instead
Thanks @aslafy-z Yes that was my starting position. And when I run helm repo add like you suggest and then run helm dep update for nextcloud-onlyoffice it works. But when I have these repos as described in my first post and I run helmfile apply, it doesn't...
You could also try to set repository key from your dep to git+https://code.greenhost.net/openappstack/nextcloud@onlyoffice-documentserver?ref=4-use-onlyoffice-instead-of-collabora instead
I tried that too in this post: https://github.com/roboll/helmfile/issues/726#issuecomment-506245134
As far as I can tell it seems to not empty it but set it to whatever is in settings.Home, but I don't really understand why that doesn't default to /root/.helm in my case
Yeah that's interesting. I see no suspicious code that can result in the empty value, at least in the recent helm code-base.
On helm dependency build, settings are parsed from flags:
https://github.com/helm/helm/blob/b0b0accdfc84e154b3d48ec334cd5b4f9b345667/cmd/helm/dependency_build.go#L78
HELM_HOME is derived from the settings.Home:
https://github.com/helm/helm/blob/7cad59091a9451b2aa4f95aa882ea27e6b195f98/pkg/plugin/plugin.go#L182
And the whole envvars should be inherited to the plugin command, in this case helm-git in:
https://github.com/helm/helm/blob/7cad59091a9451b2aa4f95aa882ea27e6b195f98/pkg/getter/plugingetter.go#L67-L68
🤔
@mrtndwrd Do you see anything suspicious when you run helm home --debug?
For my it's like this - where the first line is settings.Home:
$ helm home --debug
/Users/myuesr/.helm
Repository: /Users/myuesr/.helm/repository
RepositoryFile: /Users/myuesr/.helm/repository/repositories.yaml
Cache: /Users/myuesr/.helm/repository/cache
Stable CacheIndex: /Users/myuesr/.helm/repository/cache/stable-index.yaml
Starters: /Users/myuesr/.helm/starters
LocalRepository: /Users/myuesr/.helm/repository/local
Plugins: /Users/myuesr/.helm/plugins
Not really...
# helm home --debug
/root/.helm
Repository: /root/.helm/repository
RepositoryFile: /root/.helm/repository/repositories.yaml
Cache: /root/.helm/repository/cache
Stable CacheIndex: /root/.helm/repository/cache/stable-index.yaml
Starters: /root/.helm/starters
LocalRepository: /root/.helm/repository/local
Plugins: /root/.helm/plugins
Actually we have the same issue when using both helm-diff and helm-git plugins... Very annoying as for us git references are simply unusable right now... Any workaround for that?
Workaround
sed -i 's| \-\-skip\-refresh||g' "$(helm env HELM_PLUGINS)/helm-git/helm-git-plugin.sh"
More info https://github.com/aslafy-z/helm-git/pull/155
// upd
Dont do this! Fork bomb!
helm dep up calls recursively helm dep up for each helm repos and every time helm calls helm-git plugin where plugin calls helm dep up - and here we gotcha infinity loop.
This should now be fixed with https://github.com/aslafy-z/helm-git/releases/tag/v0.11.2. Thank you @metajiji for your work on this issue. Can you have a look @mrtndwrd?
Sorry, I can't confirm whether this solves the original issue because we've since moved to hosting our charts on Helm repositories and we've also replaced helmfile with flux2/helm-controller.
The charts and branches from this issue's description are also gone, so it would be quite a lot of work to bring them back.
Congrats on solving it!
7 Jun 2022 13:42:07 Zadkiel Aharonian @.***>:
This should now be fixed with https://github.com/aslafy-z/helm-git/releases/tag/v0.11.2. Thank you @metajiji[https://github.com/metajiji] for your work on this issue. Can you have a look @mrtndwrd[https://github.com/mrtndwrd]?
— Reply to this email directly, view it on GitHub[https://github.com/roboll/helmfile/issues/726#issuecomment-1148553013], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAGLJUZ43AJDTNTCIAGPAG3VN4YQZANCNFSM4H3RWLLA]. You are receiving this because you were mentioned.[Tracking image][https://github.com/notifications/beacon/AAGLJUYUIRIEU5ATXZFRWPTVN4YQZA5CNFSM4H3RWLLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIR2YONI.gif]