k8s
k8s copied to clipboard
alpine-docker
Make Helm Plugins available to all users
When running the image as a non-root user, the helm plugins can't be found because they are normally located at the user's home folder. During installation, they are installed to /root, which is not globally readable:
> docker run --rm -ti -u "1000:1000" alpine/k8s:1.27.4 helm unittest
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.
But if we configure the HELM_PLUGINS environment variable, Helm will install AND look for the plugins at that folder. By making the folder globally readable, we allow any user to use the plugins (without being able to modify them):
> docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:local helm unittest tests/dummy-chart
### Chart [ dummy-chart ] tests/dummy-chart
PASS images test tests/dummy-chart/tests/images_test.yaml
PASS labels test tests/dummy-chart/tests/labels_test.yaml
PASS names test tests/dummy-chart/tests/names_test.yaml
Charts: 1 passed, 1 total
Test Suites: 3 passed, 3 total
Tests: 27 passed, 27 total
Snapshot: 0 passed, 0 total
Time: 38.429632ms
This is a much simpler change than #55 but should already help with people using the image in CI/CD environments where one can't use the root user.
@ozbillwang Sorry, I noticed you gave it a :+1: but I don't know if you need anything else to merge this. If you do, feel free to share here and I'll get on it right away.
@ozbillwang Sorry, I noticed you gave it a 👍 but I don't know if you need anything else to merge this. If you do, feel free to share here and I'll get on it right away.
Thank you for the PR; it shows promise.
I'll need some time to verify the changes, so please be patient.
So the default path is /root/.cache/helm/plugins currently.
second, I didn't see the different with the Dockerfile you changed
# with exist image
$ docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:1.27.4 helm unittest .
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.
# with local image built from your Dockerfile
$ docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:local helm unittest .
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.
# and it breaks the exist usage
$ docker run --rm -ti -v "$PWD:/apps" alpine/k8s:local helm unittest
Error: unknown command "unittest" for "helm"
Run 'helm --help' for usage.
That is strange. Did you build the image with a newer version of Helm? I've noticed that the default version on the Dockerfile is too old and does not support the environment variable:
angelo@irgeshud:~/tmp > git clone [email protected]:Angelin01/k8s.git -b patch-1
Cloning into 'k8s'...
remote: Enumerating objects: 412, done.
remote: Counting objects: 100% (239/239), done.
remote: Compressing objects: 100% (83/83), done.
remote: Total 412 (delta 168), reused 186 (delta 146), pack-reused 173
Receiving objects: 100% (412/412), 74.87 KiB | 324.00 KiB/s, done.
Resolving deltas: 100% (235/235), done.
angelo@irgeshud:~/tmp > cd k8s/
# NOTICE THE BUILD ARG HERE
angelo@irgeshud:~/tmp/k8s (patch-1|u=) > docker build . -t alpine/k8s:2023-08-03 --build-arg HELM_VERSION=3.12.2 --build-arg KUBECTL_VERSION=1.27.4
[+] Building 80.9s (20/20) FINISHED docker:default
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 3.72kB 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 1.6s
=> [ 1/16] FROM docker.io/library/alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 0.0s
=> CACHED [ 2/16] RUN case `uname -m` in x86_64) ARCH=amd64; ;; armv7l) ARCH=arm; ;; aarch64) ARCH=arm64; ;; ppc64le) ARCH=ppc64le; ;; s390x) ARCH=s390x; ;; *) echo "un-supported 0.0s
=> CACHED [ 3/16] RUN . /envfile && echo $ARCH && apk add --update --no-cache curl ca-certificates bash git && curl -sL https://get.helm.sh/helm-v3.12.2-linux-${ARCH}.tar.gz | tar -xvz && mv 0.0s
=> [ 4/16] RUN mkdir -p "/helm-plugins" && chmod 755 "/helm-plugins" 0.4s
=> [ 5/16] RUN helm plugin install https://github.com/databus23/helm-diff && rm -rf /tmp/helm-* 6.4s
=> [ 6/16] RUN helm plugin install https://github.com/helm-unittest/helm-unittest && rm -rf /tmp/helm-* 5.6s
=> [ 7/16] RUN helm plugin install https://github.com/chartmuseum/helm-push && rm -rf /tmp/helm-* /root/.local/share/helm/plugins/helm-push/testdata /root/.cache/helm/plugins/https-github.co 4.5s
=> [ 8/16] RUN . /envfile && echo $ARCH && curl -sLO https://storage.googleapis.com/kubernetes-release/release/v1.27.4/bin/linux/${ARCH}/kubectl && mv kubectl /usr/bin/kubectl && chmod +x /u 3.3s
=> [ 9/16] RUN . /envfile && echo $ARCH && curl -sLO https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv3.8.1/kustomize_v3.8.1_linux_${ARCH}.tar.gz && tar xvzf kustomize 2.1s
=> [10/16] RUN . /envfile && echo $ARCH && curl -sL "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_${ARCH}.tar.gz" | tar xz -C /tmp && mv /tmp/eksctl /usr/bin 3.2s
=> [11/16] RUN apk add --update --no-cache python3 && python3 -m ensurepip && pip3 install --upgrade pip && pip3 install awscli && pip3 cache purge 21.3s
=> [12/16] RUN apk add --update --no-cache jq yq 3.7s
=> [13/16] RUN . /envfile && echo $ARCH && authenticator=$(curl -fs https://api.github.com/repos/kubernetes-sigs/aws-iam-authenticator/releases/latest | jq --raw-output '.name' | sed 's/^v//') && 10.8s
=> [14/16] RUN apk add --update --no-cache gettext 2.3s
=> [15/16] RUN . /envfile && echo $ARCH && curl -L https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.18.1/kubeseal-0.18.1-linux-${ARCH}.tar.gz -o - | tar xz -C /usr/bin/ && chm 9.2s
=> [16/16] WORKDIR /apps 1.5s
=> exporting to image 4.5s
=> => exporting layers 4.5s
=> => writing image sha256:37bf687ed8fe0a760dae05424e5fc84e418f088ee802e329706abb0f7a3e1da0 0.0s
=> => naming to docker.io/alpine/k8s:2023-08-03 0.0s
angelo@irgeshud:~/tmp/k8s (patch-1|u=) > docker run --rm -ti -u "1000:1000" -v "$PWD:/apps" alpine/k8s:2023-08-03 helm unittest .
### Error: Chart.yaml file is missing
Charts: 1 failed, 1 errored, 0 passed, 1 total
Test Suites: 0 passed, 0 total
Tests: 0 passed, 0 total
Snapshot: 0 passed, 0 total
Time: 3.060674ms
Error: plugin "unittest" exited with error
Maybe we should consider updating the default versions on the ARGs?
Also, as a separate suggestion, maybe we can introduce some smoke tests during image build to help with these base verifications. Something like this:
RUN helm plugin install https://github.com/helm-unittest/helm-unittest && \
rm -rf /tmp/helm-* && \
helm unittest --help # This command will fail if the plugin is not installed properly and break the docker build.
I can probably whip this up in another PR.
let me check, seems I build with old version of helm. The latest is:
helm version is 3.12.2
The Dockerfile need be reviewed, because several part has hardcoding the path of helm plugin and try to clean it. We need adjust them as well.
https://github.com/alpine-docker/k8s/blob/master/Dockerfile#L40-L42
I've adjusted the removal of helm-pushs test data. The other files, like /root/.cache are controlled by other environment variables and don't need to be changed here. Here's the variables/paths that helm considers as a non root user:
/apps $ helm env
HELM_BIN="helm"
HELM_BURST_LIMIT="100"
HELM_CACHE_HOME="/.cache/helm"
HELM_CONFIG_HOME="/.config/helm"
HELM_DATA_HOME="/.local/share/helm"
HELM_DEBUG="false"
HELM_KUBEAPISERVER=""
HELM_KUBEASGROUPS=""
HELM_KUBEASUSER=""
HELM_KUBECAFILE=""
HELM_KUBECONTEXT=""
HELM_KUBEINSECURE_SKIP_TLS_VERIFY="false"
HELM_KUBETLS_SERVER_NAME=""
HELM_KUBETOKEN=""
HELM_MAX_HISTORY="10"
HELM_NAMESPACE="default"
HELM_PLUGINS="/helm-plugins"
HELM_REGISTRY_CONFIG="/.config/helm/registry/config.json"
HELM_REPOSITORY_CACHE="/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/.config/helm/repositories.yaml"
and as the root user:
/apps # helm env
HELM_BIN="helm"
HELM_BURST_LIMIT="100"
HELM_CACHE_HOME="/root/.cache/helm"
HELM_CONFIG_HOME="/root/.config/helm"
HELM_DATA_HOME="/root/.local/share/helm"
HELM_DEBUG="false"
HELM_KUBEAPISERVER=""
HELM_KUBEASGROUPS=""
HELM_KUBEASUSER=""
HELM_KUBECAFILE=""
HELM_KUBECONTEXT=""
HELM_KUBEINSECURE_SKIP_TLS_VERIFY="false"
HELM_KUBETLS_SERVER_NAME=""
HELM_KUBETOKEN=""
HELM_MAX_HISTORY="10"
HELM_NAMESPACE="default"
HELM_PLUGINS="/helm-plugins"
HELM_REGISTRY_CONFIG="/root/.config/helm/registry/config.json"
HELM_REPOSITORY_CACHE="/root/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/root/.config/helm/repositories.yaml"
Hi, it's been a few days, I believe I've adjusted what needs adjusting, unless you want me to update the default versions for the ARGs or similar in this PR too. Anything else you need to merge this?