Adding repo stable ... every time

[TJM]

Hi,

We are in an "air gapped" environment, where the hosts do not have Internet access directly. We have to use artifactory, which is in a different security zone that has Internet access. I am not sure if this has anything to do with my issue below, but I think its relevant.

NOTE: Our helmfile.yaml has the following to be able to reach the "stable" repo.

repositories:
- name: stable
  url: https://artifactory.company.com/artifactory/helm-charts

Every time I run helmfile diff, helmfile apply or helmfile lint or whatever, I have several seconds (30+) waiting on this "Adding repo stable" and "Updating repo" most of it is spent during the "Adding repo" step.

$ helmfile -e sandbox diff
Adding repo stable https://artifactory.company.com/artifactory/helm-charts
"stable" has been added to your repositories

Updating repo
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈

1. Is there some way to pre-add the repo so that it doesn't have to be added every time? NOTE: I already added it to helm:

$ helm repo list
NAME  	URL
stable	https://artifactory.company.com/artifactory/helm-charts

2. Is there a way to set a TTL or something so that if it has updated a repo in the last say 5 minutes, it can skip that step? Or maybe a "--used-cached-repodata" ?

I am just trying to improve the overall experience, as in the future, it sounds like there will not be a centralized repo, there will be a bunch of separate repos, which I would hate to have to wait 30 seconds to add/update each one. :)

Tommy

[mumoshu]

@TJM Hey! Does adding --skip-deps to the command help?

[TJM]

@mumoshu It does, and that is a reasonable workaround. (slightly better than commenting the repositories section before running a series of diffs/lints)

[mumoshu]

@TJM Thanks for the confirmation!

The fundamental issues here is that Helmfile always run the time-consuming helm repo update to ensure all the repos are up-to-date. It is run because we have (currently) no way to know the helmfile.yaml being processed requires the latest chart repos or not.

Maybe we can enhance Helmfile to do chart registry updates on our own(=fetch chart registry indices from within go without calling out to the helm binary?) to make it faster.

Do you have any other idea? Any feedack is welcomed.

[TJM]

I was actually going to suggest that most of the time is spent on the "repo add" portion, so we could try to detect whether the repo was already added or not.... or better yet, we could submit a PR to helm itself to detect when a repo is already added and not add again? or at least not download the package index when a repo is added (perhaps a HEAD req?)? I think it might be a bad idea to try to bypass the helm binary.

I think, for the repo update, the best thing I can think of was some sort of TTL. So it knows if it has been run in the past (5-10-15?) minutes, it probably doesn't need to run again unless a --force-update option is included? It would be better if each repo had some sort of "cache" options (they probably already do) and helm(file) just obeyed those, maybe a HEAD req to see if some metadata has changed? (shrug)

So I am not saying we shouldn't have the latest updates (obviously depending on the contents of the helmfile), but have some way to be idempotent and not try to add a repo thats already added and update a repo thats already up to date.

For now, I can just add --skip-deps when I am running it repeatedly, for sure. :)

[max-rocket-internet]

Also hitting this issue in later versions of helmfile, it slows everything down a lot. I will use the --skip-deps option but would be nice to have a more elegant fix 🙂

[nicolaerosia]

slow here as well, I'm using the suggested workaround --skip-deps

[lucasfcnunes]

Was there a solution for this after the move to https://github.com/helmfile/helmfile?