[Bug] Submit Helm charts to Artifact repo

[arun-gupta]

Priority

Undecided

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• [X] Pull docker images from hub.docker.com
• [ ] Build docker images from source

Deploy method

• [ ] Docker compose
• [ ] Docker
• [X] Kubernetes
• [X] Helm

Running nodes

Single Node

What's the version?

latest

Description

Currently, the Helm charts are downloaded using curl or the workspace needs to be cloned. The Helm community provides a standard Artifact Repo to store such artifacts. The Helm charts should be stored in this repo providing a more natural experience.

Reproduce steps

The steps are listed at https://github.com/opea-project/GenAIExamples/blob/main/ChatQnA/kubernetes/intel/README.md and need to be simplified.

Raw log

No response

[yongfengdu]

Now we're putting all the helm charts to this repo: (Included v0.8, v0.9 releases, the 1.0 release is ongoing) https://opea-project.github.io/GenAIInfra/

[arun-gupta]

That sounds like a good experience, looking forward to test it. Will all the docs be updated to include a link to use this Helm repo?

[yongfengdu]

The helm repo doc was included in an overall README for helm charts themselves, but not linked from the portal yet. https://github.com/opea-project/GenAIInfra/tree/main/helm-charts#using-helm-charts-repository

[poussa]

Also, the manifest files which are generated from the GenAIInfra helm charts and are part of the GenAIExamples repo should be deleted. This is very confusing user experience.

For example, the GenAIInfra/ChatQnA helm chart should be enough. Now we have the generated manifest files (via helm template also checked in GenAIExamples.

This pollutes the repo with extra 50+ yaml files plus READMEs which add no value and confuses users.

[yongfengdu]

Also, the manifest files which are generated from the GenAIInfra helm charts and are part of the GenAIExamples repo should be deleted. This is very confusing user experience.

For example, the GenAIInfra/ChatQnA helm chart should be enough. Now we have the generated manifest files (via helm template also checked in GenAIExamples.

This pollutes the repo with extra 50+ yaml files plus READMEs which add no value and confuses users.

Agree, I also think the manifests are duplicate stuff for ChatQnA. More confusing is that, some of the manifests (Which have Helm chart support) are automatically generated from "helm template", but some of them are not supported by Helm chart, but also have manually maintained manifests(https://github.com/opea-project/GenAIExamples/tree/main/FaqGen/kubernetes/intel/cpu/xeon/manifest).

There should be an overall guideline of how to deploy on k8s, including code structure and everyone should follow it. After this, we can re-organize/cleanup everything, otherwise, just delete the manifests will cause more confusion.

[hshen14]

Now we're putting all the helm charts to this repo: (Included v0.8, v0.9 releases, the 1.0 release is ongoing) https://opea-project.github.io/GenAIInfra/

Is this becoming official? or https://opea-project.github.io/helm-charts better?

CC @chensuyue

[poussa]

There are two topics discussed here:

1. where the Helm chart source code is
2. where they are consumed from

About 1): Now all the Helm charts source code is in GenAIInfra which is the correct place (IMO).

About 2) This is what @arun-gupta is asking in this issue. We have basically 3 options:

1. Artifact hub (artifacthub.io)
2. GitHub pages (opea-project.github.io/GenAIInfra)
3. GitHub Container Registry (ghcr.io)

There are pros and cons in each. A lot of projects use more than one location. Quick summary below.

Artifact Hub: independent (+), well known (+), detached from the project (-) GitHub pages: old school GH way (-) ghcr.io: GH new way to host Helm charts (+), attached to the project (+), can also host container images (+), repo name needs to change since upper case letter are not allowed in OCI URIs (-)

[arun-gupta]

https://helm.sh/docs/topics/chart_repository/ provides more options about where Helm charts should be hosted.

Artifact Hub seems to be the most reasonable choice to me.

[poussa]

@arun-gupta that is the "legacy" way of hosting Helm Charts. The "new" way is based on the OCI (open container initiative).

https://helm.sh/docs/topics/registries/

All major CSPs, ghcr.io and Artifact Hub supports OCI Helm registries. I would add OPEA Helm charts to both ghcr.io and Artifact Hub. It is just few lines more in the CI/CD pipeline once we have it.

[arun-gupta]

@poussa clearly my knowledge is dated :)

lets do the most intuitive place for developers to access these Helm charts. There should be no additional step required to run these Helm charts in a developer environment,

[yongfengdu]

Published to ghcr