Document/support way of providing CA file needed to talk to image registry.

[GrahamDumpleton]

Mentioned this in https://github.com/pivotal/kpack/issues/196#issuecomment-547232594 but creating separate issue to give better visibility since that other issue was for different problem.

Issue is how can one supply a CA file fir secure SSL connection when taking to an image registry?

I can't see that this is documented, nor can I determine if it is supported in some way.

For an image registry hosted in the cluster, the normal CA used with secure connections internal to the registry may be used. Or a separate non public CA might be used. In either case, need a way of supplying the CA file in the secret with login credentials.

[matthewmcnew]

At this time we don't currently have a mechanism to provide a CA cert for the image registry.

Theoretically you could add support by adding the CA cert to a custom builder image and mounting the CA cert in the kpack controller via an init container.

[p-alexisv]

I was able to successfully get kpack to connect to a private registry (Harbor) with self-signed certs.

I had to clone two images (for ClusterBuilder and BUILD_INIT_IMAGE) and then copy the CA cert file into these (in /etc/ssl/certs) and then upload them to a registry. Then, modified the ClusterBuilder and the kpack-controller Deployment to use my custom images.

I also had to create a configmap with the CA cert data. Then, modified the kpack-controller Deployment to create a volume using that configmap and had it mounted (in /etc/ssl/certs) to the controller container.

In all 3 cases, the filename of the CA cert is in the hashed format (e.g., 9xx2x831.0) when I put it in /etc/ssl/certs.

[cameronbanowsky]

@p-alexisv : when you say you uploaded your custom images, did you use your private registry, or did you need to use a public registry at that point?

[p-alexisv]

@p-alexisv : when you say you uploaded your custom images, did you use your private registry, or did you need to use a public registry at that point?

I uploaded them to a public registry. I think you probably can use a private one if your cluster can communicate properly with it. I didn't try it though.

[cameronbanowsky]

Cool thank you @p-alexisv ... I will update this thread to show our hack to this problem

[jeffellin]

ClusterBuilder

@p-alexisv when you say ClusterBuilder Image what image are you referring to? In my case the builder never becomes ready because it can't pull from harbor. I updated the build-init-image as you described but am unsure which one I need to update for the ClusterBuilder. Which image is responsible for assembling and publishing the cluster builder? Is it paketobuildpacks/build:base-cnb ?

[mattmoor]

I'm guessing this boils down to your usage of github.com/google/go-containerregistry, which we (knative/serving) use for digest resolution. I'd wager a similar workaround to what we have documented here would work for kpack as well (though I haven't tried it): https://knative.dev/docs/serving/tag-resolution/#custom-certificates

[matthewmcnew]

@jeffellin You will also need to provide the ca cert to the kpack controller image.

You can do that by extending the image as @p-alexisv has described or by simply mounting it in the container as suggested in the knative documenation.

[cmoulliard]

You can do that by extending the image as @p-alexisv has described or by simply mounting it in the container as suggested in the knative documenation.

Is it possible to document here what it should be done to fix the Deployment k8s resource of the kpack controller in order to mount from a secret the self signed certificate please ?

Something like this

kc delete deployment/kpack-controller -n kpack
cat <<EOF | kubectl apply -n kpack -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kpack-controller
  namespace: kpack
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kpack-controller
  template:
    metadata:
      labels:
        app: kpack-controller
        version: 0.2.2-rc.1
    spec:
      serviceAccountName: controller
      nodeSelector:
        kubernetes.io/os: linux
      volumes:
        - name: custom-certs
          secret:
            secretName: cert-key
      containers:
        - name: controller
          image: gcr.io/cf-build-service-public/kpack/controller@sha256:ec256da7e29eeecdd0821f499e754080672db8f0bc521b2fa1f13f6a75a04835
          volumeMounts:
            - name: custom-certs
              mountPath: /certs
          env:
            - name: SSL_CERT_DIR
              value: /certs
            - name: CONFIG_LOGGING_NAME
              value: config-logging
            - name: CONFIG_OBSERVABILITY_NAME
              value: config-observability
            - name: METRICS_DOMAIN
              value: kpack.io
            - name: SYSTEM_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: BUILD_INIT_IMAGE
              valueFrom:
                configMapKeyRef:
                  name: build-init-image
                  key: image
            - name: BUILD_INIT_WINDOWS_IMAGE
              valueFrom:
                configMapKeyRef:
                  name: build-init-windows-image
                  key: image
            - name: REBASE_IMAGE
              valueFrom:
                configMapKeyRef:
                  name: rebase-image
                  key: image
            - name: COMPLETION_IMAGE
              valueFrom:
                configMapKeyRef:
                  name: completion-image
                  key: image
            - name: COMPLETION_WINDOWS_IMAGE
              valueFrom:
                configMapKeyRef:
                  name: completion-windows-image
                  key: image
            - name: LIFECYCLE_IMAGE
              valueFrom:
                configMapKeyRef:
                  name: lifecycle-image
                  key: image
          resources:
            requests:
              cpu: 20m
              memory: 100Mi
            limits:
              cpu: 100m
              memory: 400Mi
EOF