helm-push icon indicating copy to clipboard operation
helm-push copied to clipboard

Multiple API Sets Support

Open shizhMSFT opened this issue 7 years ago • 8 comments

ChartMuseum is a well-known open-source Helm Chart Repository with support for multiple cloud storage backends. It not only is compatible with the helm repo command set at the API level but also provides an API set for uploading new chart packages and fetching existing ones. By leveraging that uploading API, the helm-push plugin provides an easy and quick way to push chart packages to a ChartMuseum server.

As time goes by, more and more cloud service provider like Microsoft Azure, Amazon Web Service, Google Cloud, and etc., may kick in and have their own Helm Chart Repository. Since there is no standard API for uploading a chart package to the cloud, cloud service providers may have their own API sets for pushing and pulling helm chart packages with/without provenance files. It would be nice and cool for helm-push plugin to support those API sets in addition to the ChartMuseum API. With multiple API sets support, open-source software users/developers will gain smoother and more convenient experiences.

In this issue, we propose two implementation directions:

  1. Use different API sets by analyzing the host name of the repository URL. If it belongs to a known cloud service provider, the helm-push plugin uses the API set specific to that cloud service provider for uploading the chart packages. Otherwise, the plugin uses the ChartMuseum API set as the default option.
  2. Have a config file or a command line flag to use a specific API set. For instance:
$ helm push --provider=microsoft mychart-0.3.2.tgz azure
Pushing mychart-0.3.2.tgz to azure...
Done.

Personally, the proposal 1 is preferable as it is transparent to the users.

shizhMSFT avatar Sep 20 '18 16:09 shizhMSFT

\cc @yuwaMSFT2 @sajayantony

shizhMSFT avatar Sep 20 '18 16:09 shizhMSFT

Deriving from URL may not work. Cloud provider may support customized domain name. For example, myregistry.com -> myregistry.azurecr.io.

yuwaMSFT2 avatar Sep 20 '18 16:09 yuwaMSFT2

another option is to specify the API spec/provider when add plugin. The drawback is all the repositories will use the configured API spec so cannot support repos from different API specs.

yuwaMSFT2 avatar Sep 20 '18 16:09 yuwaMSFT2

@shizhMSFT @yuwaMSFT2 in order to support any provider, I think the correct thing to do here is to allow the repository to describe an HTTP spec for upload. This way, we do not need to keep adding new providers support to the plugins.

In my view, the spec for upload can have a default (chartmuseum), and another spec can be used if specified in the repository index (index.yaml).

We can make this dynamic. For example something like this in index.yaml:

serverInfo:
  uploadSpec:
    method: "POST"
    route: "/v1/helm/:name/:version"
    form:
      chart: ":chart"
      prov: ":prov"

and allow for things like :name, :version, :chart, and :prov to be dynamically populated based on chart being uploaded.

We can add additional metadata to index.yaml specifying authentication if necessary.

Once we agree on spec for this, we can submit it as a proposal for Helm v3 repo spec.

jdolitsky avatar Sep 20 '18 17:09 jdolitsky

@jdolitsky @yuwaMSFT2 Server APIs may vary and might not fit the uploadSpec.

The server API could

  1. not support POST with Content-Type: multipart/form-data. i.e. Pushing a chart package with a provenance file would be 2 separated requests.
  2. have multiple requests for uploading a single chart like pushing a layer to a registry via /v2/<name>/blobs/uploads/.

shizhMSFT avatar Sep 21 '18 07:09 shizhMSFT

Both these items can definitely be achieved somehow with the proper spec. We need something that supports all cases in index.yaml

jdolitsky avatar Sep 21 '18 08:09 jdolitsky

@jdolitsky I feel it will take time to define spec for the full detailed APIs. Also this is more a support of dynamic swagger, which may not be desirable in that:

  1. normally service would like to publish a formal swagger for their APIs as a contract, and don't change
  2. dynamic swagger can be challenging on debugging when API changes.

Also, defining the right spec format here is not easy. How to describe multiple of API versions that the server could support?

Can we have compromised solution for now: in index.yml, only specify the apiSpecType (or only proper name); and let client to define the APIs to be used per apiSpecType?

serverInfo: apiSpecType:

Later if there is agreement on providing detailed API in index.yml, we can switch to use it.

yuwaMSFT2 avatar Sep 24 '18 04:09 yuwaMSFT2

@yuwaMSFT2 sure, lets do this for now

jdolitsky avatar Sep 24 '18 04:09 jdolitsky