Ability to edit charts outside of $HELM_HOME

[gabrtv]

trafficstars

Helm requires charts be edited inside $HELM_HOME/cache or $HELM_HOME/workspace today. This makes sense as a default for casual users. However, I'd also like to edit charts out of an arbitrary VCS checkout.

For example, I'd love to edit charts out of $GOPATH/src/github.com/gabrtv/charts and not be required to edit from ~/.helm/cache/gabrtv -- but still have helm respect this as a valid repo location for helm fetch.

Do others consider this a valid use-case?

[gabrtv]

This makes me want to :shower: but seems to work for now:

$ helm repo add gabrtv [email protected]:gabrtv/charts.git
$ ln -fs ~/.helm/cache/gabrtv ~/workspace/src/github.com/gabrtv/charts

[jchauncey]

I like this idea. :+1:

[arschles]

:+1: as well. Quick proposal:

Have helm respect a $HELM_WORKSPACES environment variable, which acts as an ordered list of workspaces that helm will search when any of the following local workspace commands are run:

• search
• fetch
• edit
• info (?)

$HELM_WORKSPACES will be formatted similarly to $PATH (a : delimited list), but $HELM_HOME/cache and $HELM_HOME/workspace always implicitly come first (in that order). In @gabrtv's case, $HELM_WORKSPACES would be configured as such:

export HELM_WORKSPACES="$GOPATH/src/github.com/gabrtv/charts"

but the actual workspace search order would be $HELM_HOME/cache:$HELM_HOME/workspace:$GOPATH/src/github.com/gabrtv/charts

Feedback welcome.

[arschles]

modification to the proposal:

After talking with @jchauncey, I realized how potentially counterproductive it could be to automatically prepend $HELM_HOME/cache and $HELM_HOME/workspace to $HELM_WORKSPACES.

Instead of doing so, configure those manually in $HELM_WORKSPACES like so:

export $HELM_WORKSPACES="$HELM_HOME/cache:$HELM_HOME/workspace:$GOPATH/src/github.com/gabrtv/charts

Of course, you can change the ordering as needed. Effectively, this makes $HELM_HOME the default for casual users, and $HELM_WORKSPACES an override switch.

[bacongobbler]

I think just a way to fetch/search/install packages from a local directory would be just as useful. I've been doing the symlink trick @gabrtv mentioned in the OP for now and that works fine for my use case. The only case I'd like to have it placed elsewhere is just because we live in a Go-based world where your workspace should live in $GOPATH... Before Go this wasn't the case.

Something like helm install file:///tmp/mypkg or helm search /tmp/helm/cache/ might be useful to consider, but for now the symlink workaround works great for me.

NOTE: this is low-priority in my eyes since we got a workaround, but perhaps some external feedback would be useful as well to see how other devs are using helm.

[technosophos]

Can someone articulate exactly what problem(s) we're trying to solve?

[bacongobbler]

In my use case, my problem is the dev workflow for hacking on a chart repo. I don't want to be tied to having my text editor hardcoded to use ~/.helm as my project workspace. I'd rather have my repository somewhere else in my workspace, which is why the symlink trick works for me.

[technosophos]

But why not just export $HELM_HOME in its entirety?

I know @gabrtv use case, which is to facilitate his IDE's expectation that all code is under $GOPATH. But I'm really worried that we're making things very complicated to handle only a few edge cases that can be easily accommodated with symlinks.

Keep in mind that every piece of complexity we introduce becomes "officially supported complexity". Making pathing complex means ripple-down complexity from here forward.

For example, how much more boilerplate logic does each plugin now need in order to accommodate the proposals here? Can it be easily accomplished in a few lines of code?

In contrast, look at other projects that are opinionated about directory structure, like homebrew or git.

[arschles]

I suppose the symlink strategy is probably the simplest, and the officially supported complexity point worries me. Maybe this problem could be solved simply by adding a few commands to the docs for now

[technosophos]

Here's the issue as I understand it:

The Problem

The current structure looks like this:

$HELM_HOME
    |- config.yaml  <-- tells helm which chart repos exist
    |- cache
    |    |- charts
    |    |- foo        <--- points to my chart repo that I am editing
    |
    |- workspace

The issue at hand is how do I edit my charts on a path other than $HELM_HOME/cache/foo? Say, for example, that I want my Git checkout of foo to live in /Code/github.com/my/foo.

The compounding factor: Plugins assume that if they can get to $HELM_HOME, they can discover all repos. Because of the plugin architecture, we cannot easily pass them complex info.

Solution 1: Symlink

$ ln -s $HELM_HOME/cache/foo /Code/github.com/my/foo

Pros:

• No code changes
• Nothing gets more complicated for plugins

Cons:

• Feels clunky

Solution 2: Add path to config.yaml

Add a new entry to the config.yaml file to point to the path where the charts live:

repos:
  default: charts
  tables:
  - name: charts
    repo: https://github.com/helm/charts
  - name: foo
    path: /Code/github.com/my/foo
    repo: https://github.com/my/foo

Pros:

• The config file is an explicit indicator of where that repo lives
• This is easier than symlinking?

Cons:

• Plugins now need to parse the config.yaml to find out anything at all about repos
• We need to add tooling around managing this new path
• Helm is no longer self-contained inside of HELM_HOME

Solution 3: More Environment Variables

Another solution is to support lots of environment variables, like:

export $HELM_TABLES="$HELM_HOME/charts:/Code/github.com/my/foo
export $HELM_TABLE_NAMES="charts:foo"

Pros:

• Just more env vars

Cons:

• Represent multiple dimensions of data inside of the env vars, and leave that info to be reconstituted

Solution 4: Tooled symlink

Add a command like this:

$ helm repo link foo /Code/github.com/my/foo

And behind the scenes it does any necessary symlinking.

Pros:

• Easier than manually symlinking?

Cons:

• Dependency resolution becomes slightly more complicated
• The tooling must then take on the responsibility for error handling when target path doesn't exist

What other solutions do we have?

[technosophos]

FWIW, since I've been asked a few times now, here's how I edit charts en masse:

$ export HELM_HOME=~/Code/helm_home
$ helm repo add ts github.com/technosophos/charts
$ cd $(helm home)/cache/ts
# Do work
$ git commit
$ git push origin master

When I'm working on a clone of the master repo, I augment the above by doing this:

$ git remote add upstream UPSTREAM_URL
$ git pull --rebase upstream master # or whatever

I had apparently wrongfully assumed that this was the same workflow others were using as well. Anyone else want to share their workflow for working on a big charts repo?

[bacongobbler]

@technosophos same workflow here, only I use helm repo add bacongobbler [email protected]:bacongobbler/charts instead. Afterwards, I add a symlink to $GOPATH/src/github.com/bacongobbler/charts just so I can work in my text editor. Solution 1 works for me.