k3s

k3s copied to clipboard

Environmental Info: K3s Version: Latest

Node(s) CPU architecture, OS, and Version Docker, alpine:3.15 and Scratch image.

Cluster Configuration: I didn't manage to run it, I'm stuck in the build process.

Describe the bug:

I've tried to build the k3s from this artifact in my own Dockerfile definition. The Dockerfile looks like this but with small changes. Here is the changed Dockerfile (which uses the artifact not the build folder)

FROM alpine:3.15 as base
RUN apk add -U ca-certificates tar zstd curl
RUN curl -L https://github.com/k3s-io/k3s/releases/download/v1.24.3%2Bk3s1/k3s-airgap-images-amd64.tar.zst -o /data.tar.zst
RUN mkdir -p /image/etc/ssl/certs /image/run /image/var/run /image/tmp /image/lib/modules /image/lib/firmware && \
    tar -xa -C /image -f /data.tar.zst && \
    cp /etc/ssl/certs/ca-certificates.crt /image/etc/ssl/certs/ca-certificates.crt

FROM scratch
ARG VERSION="dev"
COPY --from=base /image /
RUN mkdir -p /etc && \
    echo 'hosts: files dns' > /etc/nsswitch.conf && \
    echo "PRETTY_NAME=\"K3s ${VERSION}\"" > /etc/os-release && \
    chmod 1777 /tmp
VOLUME /var/lib/kubelet
VOLUME /var/lib/rancher/k3s
VOLUME /var/lib/cni
VOLUME /var/log
ENV PATH="$PATH:/bin/aux"
ENV CRI_CONFIG_FILE="/var/lib/rancher/k3s/agent/etc/crictl.yaml"
ENTRYPOINT ["/bin/k3s"]
CMD ["agent"]

The only change is in line no 3 where I add a curl to download the artifact Steps To Reproduce:

• Installed K3s: Latest
• Create new Dockerfile with content that is already written above.
• Try to build the Dockerfile with: docker build .

Expected behavior: The expected behaviour is to have the ability to compile the Dockerfile from already build artifact.

Actual behavior: The build fails with logs which is written below.

Additional context / logs:

[+] Building 39.8s (9/9) FINISHED                                                                                                                                                                               
 => [internal] load build definition from Dockerfile                                                                                                                                                   0.0s
 => => transferring dockerfile: 920B                                                                                                                                                                       0.0s
 => [internal] load .dockerignore                                                                                                                                                                          0.0s
 => => transferring context: 34B                                                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/alpine:3.15                                                                                                                                             0.0s
 => [base 1/4] FROM docker.io/library/alpine:3.15                                                                                                                                                          0.0s
 => CACHED [base 2/4] RUN apk add -U ca-certificates tar zstd curl                                                                                                                                         0.0s
 => [base 3/4] RUN curl -L https://github.com/k3s-io/k3s/releases/download/v1.24.3%2Bk3s1/k3s-airgap-images-amd64.tar.zst -o /data.tar.zst                                                                37.4s
 => [base 4/4] RUN mkdir -p /image/etc/ssl/certs /image/run /image/var/run /image/tmp /image/lib/modules /image/lib/firmware &&     tar -xa -C /image -f /data.tar.zst &&     cp /etc/ssl/certs/ca-certif  1.3s
 => [stage-1 1/2] COPY --from=base /image /                                                                                                                                                                0.6s
 => ERROR [stage-1 2/2] RUN mkdir -p /etc &&     echo 'hosts: files dns' > /etc/nsswitch.conf &&     echo "PRETTY_NAME="K3s dev"" > /etc/os-release &&     chmod 1777 /tmp                                 0.1s
------
 > [stage-1 2/2] RUN mkdir -p /etc &&     echo 'hosts: files dns' > /etc/nsswitch.conf &&     echo "PRETTY_NAME="K3s dev"" > /etc/os-release &&     chmod 1777 /tmp:
#9 0.141 runc run failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory
------
executor failed running [/bin/sh -c mkdir -p /etc &&     echo 'hosts: files dns' > /etc/nsswitch.conf &&     echo "PRETTY_NAME=\"K3s ${VERSION}\"" > /etc/os-release &&     chmod 1777 /tmp]: exit code: 1

This is a problem with your dockerfile:

FROM scratch
ARG VERSION="dev"
COPY --from=base /image /
RUN mkdir -p /etc && \
    echo 'hosts: files dns' > /etc/nsswitch.conf && \
    echo "PRETTY_NAME=\"K3s ${VERSION}\"" > /etc/os-release && \
    chmod 1777 /tmp

 => ERROR [stage-1 2/2] RUN mkdir -p /etc &&     echo 'hosts: files dns' > /etc/nsswitch.conf &&     echo "PRETTY_NAME="K3s dev"" > /etc/os-release &&     chmod 1777 /tmp
#9 0.141 runc run failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory

You can't RUN arbitrary shell command in a scratch image -- there is no shell for the commands to be run in.

@olix0r - How is here possible https://github.com/k3s-io/k3s/blob/master/package/Dockerfile#L8? My Dockerfile is the same as the one used in the k3s project. How can it work there? cc: @brandond

@MihaiSandor Good question! I'm not a k3s maintainer, but I can try to figure this out with you :)

Re: my original answer, we can confirm that with a simple Dockerfile like:

#; 
:; cat Dockerfile 
FROM scratch
RUN echo hello
#; 
:; docker build --progress=plain .
#1 [internal] load build definition from Dockerfile
#1 sha256:74ed860ffe7a61c2371204b920cd81e593f91c77e05171bb2f59f43db728df99
#1 transferring dockerfile: 107B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:85569fd5b3e0c94d8302fe7a5b305fb88d0c21a4d6ac92585e413f3504cbb483
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [1/1] RUN echo hello
#3 sha256:631cad4b12396019c51be488d27ebc2db83d8bc2362a7ee6d1e0d359f0007948
#3 0.254 runc run failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory
#3 ERROR: executor failed running [/bin/sh -c echo hello]: exit code: 1
------
 > [1/1] RUN echo hello:
------
executor failed running [/bin/sh -c echo hello]: exit code: 1

I'm guessing, though, that the k3s dockerfile works because the /image directory includes a /bin/sh: https://github.com/k3s-io/k3s/blob/c3f830e9b9ed8a4d9d0e2aa663b4591b923a296e/package/Dockerfile#L10

Your example builds the image directory from

RUN curl -L https://github.com/k3s-io/k3s/releases/download/v1.24.3%2Bk3s1/k3s-airgap-images-amd64.tar.zst -o /data.tar.zst

Looking at the output of curl -L https://github.com/k3s-io/k3s/releases/download/v1.24.3%2Bk3s1/k3s-airgap-images-amd64.tar.zst | tar --zstd -tvf -, it looks like this file is a docker image -- it contains a bunch of layers like

-rwxr-xr-x 0/0               0 2022-06-13 21:19 f280b376d719a13dc647553ad9d8446781cea0b54b18680ffd7d7c1ccdaa9642/
-rw-r--r-- 0/0               3 2022-06-13 21:19 f280b376d719a13dc647553ad9d8446781cea0b54b18680ffd7d7c1ccdaa9642/VERSION
-rw-r--r-- 0/0             477 2022-06-13 21:19 f280b376d719a13dc647553ad9d8446781cea0b54b18680ffd7d7c1ccdaa9642/json
-rw-r--r-- 0/0        17502208 2022-06-13 21:19 f280b376d719a13dc647553ad9d8446781cea0b54b18680ffd7d7c1ccdaa9642/layer.tar

I'm assuming further processing is required to get this into a state where the /image directory includes the files you need.

This repository uses a bot to automatically label issues which have not had any activity (commit/comment/label) for 180 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the bot can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the bot will automatically close the issue in 14 days. Thank you for your contributions.

I'm not sure what's going on here, but you can't build k3s from the airgap image tarball. That contains all the other docker images required by k3s to run its pods. It does not contain k3s itself.