Error pulling private images for nodes deployed by KNE in kind cluster

[ashutshkumr]

When,

• deploying a KNE topology consisting of OTG and arista
• both OTG and arista images are posted to private registry
• using a kind cluster

There's currently no way to specify the registry secret for arista (or any other images deployed by KNE). For ixia-c, creating secret called ixia-pull-secret works.

I'm wondering, may be we should define the name of secret expected by KNE as well ? And may be, mandate vendor-specific controllers to use the same name for secret ?

Another issue is, pull secrets are only visible for a given namespace (at least by default). So ideally one would have to create the secret before topology creation, using the same namespace used by the topology.

Need more thoughts on this. Meanwhile, I unblocked my internal work with kind cluster using this change: https://github.com/google/kne/tree/kne-pull-secret

These are the set of commands I execute currently to pull images from private registry:

# ${1} => namespace of KNE topology
set_pull_secrets() {
    inf "Setting pull secrets for name space ${1} ..."
    kubectl create ns ${1} \
    && kubectl create secret -n ${1} docker-registry kne-pull-secret \
        --docker-server=us-central1-docker.pkg.dev \
        --docker-username=_json_key \
        --docker-password="$(cat ${GCLOUD_SVC_ACC_KEY})" \
        --docker-email=${GCLOUD_EMAIL} \
    && kubectl create secret -n ${1} docker-registry ixia-pull-secret \
        --docker-server=us-central1-docker.pkg.dev \
        --docker-username=_json_key \
        --docker-password="$(cat ${GCLOUD_SVC_ACC_KEY})" \
        --docker-email=${GCLOUD_EMAIL}
}

[alexmasi]

Looks like the private registries that you want to provide kind automatic access to are Google Artifact Registries, in this case try the option in the deployment yaml. https://github.com/google/kne/blob/f3c905c69b52d8c02a5b1f47d808c75c24c89144/deploy/deploy.go#L219

This option uses your access token (from gcloud auth print-access-token) and applies across the entire cluster.

Let me know if this works for you.

cluster:
  kind: Kind
  spec:
    name: kne
    version: ...
    image: ...
    ...
    googleArtifactRegistries:
      - us-central1-docker.pkg.dev

[ashutshkumr]

Thanks @alexmasi will look into this, but on the surface, pushing the keys to each node and restarting kubelet seems like a hack.

Moreover, private registries won't be limited to GAR/GCR. But then I also agree that there's always a workaround to pull images on the host and load them on kind nodes.

I wish there was a way to just create a secret in default namespace and refer to it from any namespace (created by KNE for topology).

[alexmasi]

The GAR solution was taken from the official kind docs: https://kind.sigs.k8s.io/docs/user/private-registries/

There is some more info there on private registry access in general