vllm-project
Replace our cloned 3rd-party yamls with helm charts
🚀 Feature Description and Motivation
currently, we manage envoy-gateway and kuberay manifest in a downstream way. It could be outdated or lose track in future. Kustomize now supports to include helm charts in the manifest file. If this way works fine and we can easily customize the image we'd like to use (we did have some patch in the image). then we deprecated the previous naive way.
# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
helmCharts:
- name: mergestat
repo: https://helm.mergestat.com/
releaseName: mergestat
namespace: admin
version: 0.1.0
valuesFile: values.yaml
Note: check the kustomize version. kubectl kustomize might not provide such features comparing to separate kustomize cli. (higher version)
Use Case
No response
Proposed Solution
No response
Research on Potential Kustomize Version Issues
Kustomize added support for helm charts in v4.1.0.
My current kubectl client (found with kubectl version) is built with kustomize v5.5.0.
Our CI script release-build.yaml downloads the latest kustomize version which is currently v5.6.0. NOTE: this should be pinned to a version, not allowed to download latest, I will fix that.
TLDR: Helm supported for both kubectl and CI versions
Another Potential Issue Regarding Usage
The docs for Kustomize state:
Although the helm related fields discussed above are handy for experimentation and development, it's best to avoid them in production.
The same argument applies to using remote git URL's in other kustomization fields. Handy for experimentation, but ill-advised in production.
It's irresponsible to depend on a remote configuration that's not under your control. Annoying enablement flags like '--enable-helm' are intended to remind one of a risk, but offer zero protection from risk. Further, they are useless are reminders, since annoying things are immediately scripted away and forgotten, as was done above in the kustomizeIt shell function.
Don't use remote configuration that you don't control in production.
Maintain a local, inflated fork of a remote configuration, and have a human rebase / reinflate that fork from time to time to capture upstream changes.
My Thoughts
Personally, I don't see an issue with this but will wait for others to weigh in before starting the changes.
Helm charts can be pinned to specific versions, and released versions aren't just going to get modified. Everybody downloads plenty of containers from remotes that they themselves don't control, I don't see why guidance should change for released charts.
Reinflation is not without its downsides, it adds manual work for maintainers rather than just bumping a version in the kustomization.yaml.
@Jeffwan @varungup90 any thoughts?
@jolfr I am coming late for this issue.
I agree with you. Remote helm chart is pinned as well. No difference between containers and such artifacts. I did see similar issues happened in python official container image https://github.com/vllm-project/aibrix/issues/959 as well.
What's more, we build good connection with all the dependencies we are using. Even there're issues, we can contact them to build patch soon. I do not see risks there.
The only think I feel not convenient is if user like to do some customizations, such as image replacement or adding more annotation etc, seems helm is not that convenient compared to kustomization.
A side problem is whether we like to provide a giant helm to include everything for people to use.