Proxy registries

[mate4st]

Description of the feature you would like to add / User story

As a user I would like to use a proxy registry in order to have control of the accessed images.

Because proxy registries forward to an upstream registries and not a mirror registries the coredns path breaks when using the overwriteRegistry feature.

Mirror registries are not being utilized because they fallback to the default registry which is a problems due to security reasons. Because it is only a proxy the image of corends cannot be just pushed and adjusted.

Solution details

For coredns to work we need to set imageRepository for coredns in the ClusterConfiguration of kubeadm.

Alternative approaches

Consider mirror registries where it would not fallback to the original registry. Maybe there is an option to control the behavior.

[mate4st]

xref: https://github.com/kubermatic/kubeone/issues/2076

[kron4eg]

Mirror registries are not being utilized because they fallback to the default registry which is a problems due to security reasons

This is not true. For example with the following config will leak nothing to outside, and all image download requests will land to the proxy.

containerRuntime:
  containerd:
    registries:
      "*":
        mirrors:
        - my.corp/all
      k8s.gcr.io:
        mirrors:
        - my.corp/k8s-proxy
      registry.k8s.io:
        mirrors:
        - my.corp/k8s-proxy
      docker.io:
        mirrors:
        - my.corp/dockerio-proxy

For example coupled together with:

registryConfiguration:
  overwriteRegistry: my.corp/system

will give stable image address for ALL kubeone managed images, that are subject to retag.

Example list of kubernetes specific images including coredns:

my.corp/system/kube-apiserver:v1.21.14
my.corp/system/kube-controller-manager:v1.21.14
my.corp/system/kube-scheduler:v1.21.14
my.corp/system/kube-proxy:v1.21.14
my.corp/system/pause:3.4.1
my.corp/system/etcd:3.4.13-0
my.corp/system/coredns:v1.8.0

[kron4eg]

For coredns to work we need to set imageRepository for coredns in the ClusterConfiguration of kubeadm.

We do set it to the overwriteRegistry value

[kron4eg]

Closing due to there is nothing to implement here.

[qeqar]

For coredns to work we need to set imageRepository for coredns in the ClusterConfiguration of kubeadm.

We do set it to the overwriteRegistry value

The overwriteRegistry will also overwrite the path, so from docker.io/coredns/coredns to my.corp/coredns.

That will make all virtual docker proxys with multiple upstreams unusable, as it retains the original path structure. For most system pods is is working as they are directly in / , but all pods like coredns (at the moment the only one) will not.

[xmudrii]

This is still not fixed. /reopen

[kubermatic-bot]

@xmudrii: Reopened this issue.

In response to this:

This is still not fixed. /reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.