support adding private registry through k0s configuration file

[mikhail-sakhnov]

Is your feature request related to a problem? Please describe.

As a user I want to be able to pull images from my own hosted registry secured by auth.

To support it currently user needs to manually follow containerd documentation and add certificates.

https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration

Describe the solution you would like

Would be great if we introduce k0s config section related to the cri settings which will allow to easy setup certificates for private registry

Describe alternatives you've considered

Manual setup.

Additional context

Request for corresponding guide appeared in the k0s slack channel couple of times.

[mikhail-sakhnov]

https://mrzik.medium.com/how-to-configure-private-registry-for-kubernetes-cluster-running-with-containerd-cf74697fa382

some configuration examples

[martencassel]

I think i can do this one.

Does this configuration layout looks ok for the very simple case of one registry ?

runtime:
  containerd:
    registry:
      server: "https://private-registry.com"
      ca: <base64 encoded ca>
      client: <base64 encoded client cert>
      key: <base64 encoded client key>

Is supporting a single registry configuration enough ? Containerd support defining multiple registries.

Do you want also to support insecure registry settings ?

[mikhail-sakhnov]

@martencassel let's go with multiple registries.

Something like

spec:
  runtime:
   registries:
    - name: docker.io:
      server: "https://docker.io"
      capabilities:
        - "pull"
      skip_verify: true
      ca: <base64 encoded ca>
      client: <base64 encoded client cert>
      key: <base64 encoded client key>
      credential:
        username: "admin"
        password: "password"
    - name: gcr.io:
      server: "https://gcr.io"
      capabilities:
        - "pull"
      skip_verify: true
      ca: <base64 encoded ca>
      client: <base64 encoded client cert>
      key: <base64 encoded client key>
    ghcr.io:
      server: "https://ghcr.io"
      capabilities:
        - "pull"
      skip_verify: false
      ca: <base64 encoded ca>
      client: <base64 encoded client cert>
      key: <base64 encoded client key>
—

I like your idea of having separate runtime section in the config, because we might want to have some other runtime related settings been introduced as well soon 👍

[martencassel]

I need to find a good name for the base directory in /var/lib/k0s/ to store all the containerd configuration directores, The containerd docs uses "certs.d" as the name.

Currently the layout is

/var/lib/k0s/certs.d/
├── registry1
│   ├── ca.pem
│   ├── client.key
│   ├── client.pem
│   └── host.toml
└── registry2
    ├── ca.pem
    ├── client.key
    ├── client.pem
    └── host.toml

[twz123]

Is this still a thing, given that k0s supports drop-in containerd confg snippets nowadays? However, the documentation could include a section on how to set up some registries in containerd with this feature.