knative
implement trusting a CA bundle
If we are using cert-manager, we currently build our trust on reading the ca.crt in the generated certificate's Secret to configure trust in:
- activator
- net-* implementation
Digging into the docs and asking our cert-manager team, they strongly recommend to use trust-manager to distribute a trust bundle decoupled from the actual secret generation. With that in place, it enables to rotate the CA without breaking the trust:
- The administrator, manually adds the old and new CA to the trust-bundle
- trust-manager distributes it to all workload
- The administrator updates the CA, all certificates are recreated when they are expired or when they are force-renewed
- The administrator removes the old CA from the trust-bundle
Note the relevant bits from the docs
Say you have a cert-manager Issuer which has the root certificate you want to trust in ca.crt. It's tempting to use the Secret directly and point at ca.crt, but a best practice would be to copy that root into a separate ConfigMap (or Secret). The reason is - as with many TLS gotchas - certificate rotation. If you rotate your issuer such that it's issued from a new root certificate, trust-manager will see the Secret be updated and automatically update your trust bundle to include the new root - immediately distrusting the old root. That means that if any services were still using a certificate issued by the old root, they'll be distrusted and will break.
To not limit the use to just trust-manager, we can use the API of a ConfigMap containing one or more PEM formatted certificates as trust-bundle sources. This also works with other approaches like OpenShift PKI or OpenShift Service Signer which have the same interface, see https://github.com/ReToCode/knative-encryption/blob/main/8-trust-sources/README.md. In the future we can also add https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3257-cluster-trust-bundles as a possible extension or replacement.
Tasks
- [x] net-kourier -> https://github.com/knative-extensions/net-kourier/pull/1171
- [x] net-istio --> we don't need that in istio, as it has its own trust mechanism already
- [ ] activator -> https://github.com/knative/serving/pull/14717
- net-gateway-api --> not possible yet
- net-contour --> not possible yet
- [x] networking (config only) -> https://github.com/knative/networking/pull/900
Some initial investigation results here: https://github.com/ReToCode/knative-encryption/tree/main/8-trust-sources#trust-manager.
Having CA Bundles in a Configmap is pretty common. OpenShift also has solutions that work this way. All of them allow to add a custom label to filter on them. So it might be a good approach to not limit the implementation to trust-manager, but instead to just read Configmaps with a specific label having PEM based bundles in them.
Also there's something coming in Kubernetes
https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3257-cluster-trust-bundles
