Describe the feature

There are cases where the cluster admin wants to share secrets with the tenant owners. Such secrets can be used, for example, by tenant owners to access cluster services like a private Image Registry or Helm Repo. Multiple secrets can be assigned for each tenant, so the cluster admin can create different secrets for each tenant.

The Capsule Operator propagates such secrets to all namespaces within the tenant. The tenant owners are allowed to delete these secrets but the controller recreates immediately. All supported type of secrets should be available:

-   docker-registry Create a secret for use with a Docker registry
-   generic         Create a secret from a local file, directory or literal value
-   tls             Create a TLS secret 

For example:

apiVersion: capsule.clastix.io/v1alpha1
kind: Tenant
metadata:
  name: oil
spec:
  owner: # required
    name: alice
    kind: User
  sharedSecrets:
    - name: registry-credentials
       type: docker-registry
       data:
         docker-server: 'registry.clastix.io'
         docker-username: 'robot$oil+credentials'
         docker-password: 'CB753sRsmjbD1qhdKwdVsSewXBcW7SVR'
         docker-email: '[email protected]'
   - name: other-secret
      type: generic
      data:
          key1: supersecret
          key2: verysupersecret

We should evaluate if provide the secret data in clear as above or already base64 encoded.

What would the new user story look like?

1. The cluster admin provisions a private Registry in the cluster
2. Each tenant owner wants to access the registry for pull and push of images using their credentials assigned by the cluster admin.
3. The cluster admin creates tenant manifests with such secret
4. The tenant owner logs to the cluster and creates a bounce of namespaces
5. The Capsule controller propagates the secret in all the namespaces and places an annotation to the namespaces reporting the name and type of the secret.
6. The tenant owner uses such secret to push and pull images from the registry

Expected behavior

Leave the cluster admin to share secrets with tenant owners

Rather than declaring secrets in a such opinionated way, I'd say we could take advantage of the ObjectReference API, although we would need a new polling controller in order to replicate changes across the Tenant's namespaces.

But honestly I'm thinking we could reuse the super capabilities provided by the Hierarchical Namespace Controller to replicate all the available resources in a specific target Namespace, I need to check the code and investigate it.

Going to spam their Slack workspace 😏

Going to add a +1 to this feature - I'm running into this right now, whereby we are automation project creation in Harbor as cluster admins when creating a tenant, and want to create the pull secret in such a way that it can be used easily within the tenant namespaces.

Don't know if it helps but I use Kubed to manage these aspects in my cluster. I create secrets in the Kubed namespace and use clastix annotations to replicate them in tenant namespaces.

@GlassOfWhiskey thanks, that's really interesting.