Kubenab

What does Kubenab do?

Kubenab is Kubernetes Admission webhook to enforce pulling of docker images from private registry.

Prerequisites

Kubernetes 1.12.0 or above with the admissionregistration.k8s.io/v1beta1 API enabled. Verify that by the following command:

kubectl api-versions | grep admissionregistration.k8s.io/v1beta1

The result should be:

admissionregistration.k8s.io/v1beta1

In addition, the MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers should be added and listed in the correct order in the admission-control flag of kube-apiserver.

Build and Push Kubenab Docker Image

# Build docker image
docker build -t my-registry/kubenab:0.3.3 .

# Push it to Docker Registry
docker push my-registry/kubenab:0.3.3

Create Kubernetes Docker registry secret

# Create a Docker registry secret called 'regsecret'
kubectl create secret docker-registry regsecret --docker-server=${DOCKER_REGISTRY} --docker-username=${DOCKER_USER} --docker-password=${DOCKER_PASS} --docker-email=${DOCKER_EMAIL}

Note: Create Docker registry secret in each non-whitelisted namespaces.

Generate TLS Certs for Kubenab

./tls/gen-cert.sh

Get CA Bundle

./webhook/webhook-patch-ca-bundle.sh

Note: You can skip this step and use Helm chart install with TLS certs generated by the cert-manager

Deploy Kubenab to Kubernetes with self generated TLS Certs

• Deploy using kubectl

# Run deployment
kubectl create -f deployment/kubenab-deployment.yaml

# Create service
kubectl create -f  deployment/kubenab-svc.yaml

• Deploy using Helm Chart

helm install --name kubenab --set docker.registrySecret=regsecret,docker.registryUrl=jfrog,whitelistNamespaces="kube-system,default",whitelistRegistries="jfrog",tls.secretName=kubenab-certs chart/kubenab/

Configure MutatingAdmissionWebhook and ValidatingAdmissionWebhook

Note: Replace ${CA_BUNDLE} with value generated by running ./webhook/webhook-patch-ca-bundle.sh

# Configure MutatingAdmissionWebhook
kubectl create -f webhook/kubenab-mutating-webhook-configuration.yaml

Note: Use MutatingAdmissionWebhook only if you want to enforce pulling of docker image from Private Docker Registry e.g JFrog Artifactory. If your container image is nginx then Kubenab will append REGISTRY_URL to it. e.g nginx will become jfrog/nginx

# Configure ValidatingAdmissionWebhook
kubectl create -f webhook/kubenab-validating-webhook-configuration.yaml

Note: Use ValidatingAdmissionWebhook only if you want to check pulling of docker image from Private Docker Registry e.g JFrog Artifactory. If your container image does not contain REGISTRY_URL then Kubenab will deny request to run that pod.

Deploy Kubenab to Kubernetes with cert-manager generated TLS Certs

• Deploy using Helm Chart

helm install --name kubenab --set docker.registrySecret=regsecret,docker.registryUrl=jfrog,whitelistNamespaces="kube-system,default",whitelistRegistries="jfrog" chart/kubenab/

ValidatingAdmissionWebhook is enabled by default, use mutatingWebhook.enabled="true" to enable MutatingAdmissionWebhook.

Test Kubenab

# Deploy nginx
kubectl apply -f test/nginx.yaml

Benchmark

Run the benchmark.sh Script in benchmark/, you only need go, docker and openssl.

Benchmark Results

==> Mutate Webhook


 1000000 / 1000000 [=====================================================================================================================] 100.00% 6850/s 2m25s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      6906,14     612,30    7957,42
  Latency      285,93us   171,41us    27,91ms
  Latency Distribution
     50%   257,00us
     75%   309,00us
     90%   376,00us
     95%   440,00us
     99%   764,00us
  HTTP codes:
    1xx - 0, 2xx - 100000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    15.05MB/s


==> Validate Webhook


 1000000 / 1000000 [=====================================================================================================================] 100.00% 6669/s 2m29s
Done!
Statistics        Avg      Stdev        Max
  Reqs/sec      6675.83     515.18    8017.83
  Latency      295.81us    99.07us    22.86ms
  Latency Distribution
     50%   267.00us
     75%   319.00us
     90%   388.00us
     95%   453.00us
     99%   759.00us
  HTTP codes:
    1xx - 0, 2xx - 1000000, 3xx - 0, 4xx - 0, 5xx - 0
    others - 0
  Throughput:    16.20MB/s

ATTENTION: This Benchmark was done on an non optimized Laptop! (Intel(R) Core(TM) i5-3230M CPU @ 2.60GHz ; 8GB RAM)