Ship lock upstream version

[nictrix]

I believe this may be a feature request.

Possibly in the short-term, we could create some documentation to help others with this similar situation.

Expected behavior

When I run ship update - I'd like to update the Helm Chart values or Kustomize overlays/patches without updating to the latest upstream's version.

Actual behavior

I don't think Replicated Ship can do this right now, though if I started with a GIT URL that had the specific version/commit in it then it would be locked. However, the question becomes how do I unlock/upgrade it then?

I can edit the file .ship/state.json and change the upstream GIT URL to lock it again. I think a command line argument could make it more straightforward.

I tried using a go-getter link, but that came back with the latest commit reference instead of the one I asked for, it did complete the entire process though:

ship init github.com/helm/charts/stable/elasticsearch?ref=d5c9a8600f7927ecd229111cf28224464b862f4d

# tried with git:: in front of it too

Another idea is to use semantic versioning with upstream Helm Charts to specify which version you are willing to go to during an update. (possibly needing to use the Helm HTTP resource instead of the Git Helm repo since it doesn't have tags)

I tried doing this:

ship init https://kubernetes-charts.storage.googleapis.com/elasticsearch-1.14.1.tgz --resource-type=helm

But that failed because it autodetected as k8s instead of using the specific resource-type of helm - Not sure if --resource-type is supposed to override the autodetected one or not?

Information

• Ship version: Latest
• Docker version: Any
• Command line run: ship update

Steps to reproduce the behavior

Currently running ship update and if there is an upstream change it takes it in.

---

One of my driving reasons for this feature is to make sure aship update doesn't break the existing deployment when using 3rd party Helm Charts. A breaking scenario would be using an existing Helm Chart which goes from application's version 1 to 2; which may be fine if I use image tag values to lock the application's container version. However, the Helm Chart itself may need to change for that version change and cause me deployment problems during the ship update CI process.

If there is a better way to think through this, where this feature would not be needed, please let me know.

[dexhorthy]

Hi @nictrix! Thanks for the detailed write up.

I think you're right, this is a great feature request that I'd like to see as well. I think there's two or three pieces here:

1. support for using ship update to track a semver string like 1.x or ^1.2.2
2. Support for ship init upstreams that are Chart tarballs
3. (maybe) Support for forcing a --resource-type (not sure about this one yet, seems that detecting a helm tar.gz is the more robust solution to this particular problem)

More details below, let's get some input from @Rob0h / @laverya / @marccampbell on whether we want to split these into their own issues or track them all here. I think any of these would be good first issues, although the first is probably a bit bigger in scope than the other two.

Semver

I believe there are some plans to address this concern of Chart which goes from application's version 1 to 2 via supporting semver in an upcoming release, but I don't know what the timeline is.

Right now you can ship init with a git sha or tag, e.g.

ship init https://github.com/helm/charts/tree/df1b1f3508abd885433e151d2019aba177dce495/stable/elasticsearch

or

ship init https://github.com/istio/istio/tree/1.0.5/install/kubernetes/helm/istio

Ideally ship can eventually support something similar for SemVer, along the lines of

ship init https://github.com/istio/istio/tree/1.x/install/kubernetes/helm/istio

Resource Type

It appears that using the --resource-type flag to specify a helm vs k8s application is not fully supported today, as you pointed out. This looks like an oversight that should definitely get tracked somewhere. A quicker path to success may be to ensure we properly detect that helm tar.gz chart format.

[laverya]

I think one thing we may need to do to solve this in the general case is to cache the upstream (& possibly include it in the state file) for future updates. A git sha might not exist in the future, and there isn't really a way to version the contents of a URL anyways. This would greatly increase state file size though, which may not be desirable.

[dexhorthy]

Following up on this @laverya -- I think right now when you do ship init github.com/istio/chart/tree/v.1.1.0/some/chart/path, it gets translated ship init github.com/istio/chart/tree/__latest__/some/chart/path.

If we were to implement proper locking, that would technically constitute a change in behavior from what folks may have come to expect.

That said I think usage volume there could be limited and this would be a welcome addition. Unclear whether this necessarily needs to be in the "breaking changes to include in 1.0" milestone.

[laverya]

Right, but what I suggest is for the behavior now to stay the same - if you run ship update without a --lock flag (or something - perhaps we add a new command ship edit instead of further overloading ship update) you still get the latest, but if you run it with that flag you use the cached version of the upstream from the last time you updated. I wouldn't consider that a breaking change...

[dexhorthy]

Yeah having it be opt in makes this really easy to get started solving.

Off the top of my head, ship update --lock makes sense to me, as does ship edit. There is probably a deeper devx conversation to have here. I think you're right that we should avoid overloading where we can.

[dexhorthy]

what happens if I ship init github.com/istio/chart/tree/v.1.1.0/some/chart/path in the lock-capable world? Do I still get auto-updated to v1.2.0 if it exists?

[laverya]

I believe the lock-capable world shouldn't change any existing behavior, so yes

[Timoses]

What about a simple --tag and --branch argument for ship init and ship update?

• Using ship init <GIT-URL> --branch XYZ would fetch latest contents from that branch.
• Subsequent ship update would check for most recent commint within that branch.
• Using ship update --branch XYZ or ship update --tag XYZ would move the tracked "git-version" to XYZ branch/tag.

--tag could support v1.* for major release locking and v1.2.* for only bugfix releases.

The above purely talks about the "git version" workflow, as "branch" and "tag" is git terminology. And so, if --tag or --branch were to be used with a URL that is not a git repository, then it would error out (e.g. "Unable to use --tag for non-git repository URL").

For following helm chart versions it could be as simple as ship init <HELM-URL> --version XYZ.

I think using --tag, --branch and --version would bring more flexibility than a --lock argument.