Create cluster in a different namespace than the operator

[kernfeld-cockroach]

This issue originally created by @Martin-Hogge as https://github.com/cockroachdb/cockroach/issues/65180 and moved here by me. @johnrk could you advise on where this sits or doesn't sit on our road map?

Describe the problem

I created the operator on a namespace different than default and I wanted to create a cluster on a dedicated namespace as well but it seems that the operator is not reacting to the creation of the manifest if is not created in the same namespace of the operator.

To Reproduce

1. Create a namespace: kubectl create ns cockroach
2. Apply crdb: kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach-operator/master/config/crd/bases/crdb.cockroachlabs.com_crdbclusters.yaml -n cockroach
3. Download the operator's manifest at https://raw.githubusercontent.com/cockroachdb/cockroach-operator/master/manifests/operator.yaml
4. Update every namespace: default by namespace: cockroach
5. Install the operator: kubectl apply -f operatior.yaml -n cockroach
6. Create another namespace: kubectl create ns app-1
7. Download the cluster manifest example at https://raw.githubusercontent.com/cockroachdb/cockroach-operator/master/examples/example.yaml
8. Update the name to be cockroachdb-app-1 and update the namespace to app-1
9. Install the cluster: kubectl apply -f example.yaml -n app-1

Nothing happens and no logs inside the operator.

Expected behavior A new cluster is created in the namespace app-1

Environment:

• CockroachDB version: v20.2.8 / Kubernetes (GKE)

[chrislovecnm]

Can we get the yaml files you used?

[protosam]

If I understand this correctly, he's trying something similar to this. I expected this to work as well, but it doesn't. The cockroachdb operator is only watching the namespace it's deployed in.

$ kubectl apply -f https://raw.githubusercontent.com/cockroachdb/cockroach-operator/master/config/crd/bases/crdb.cockroachlabs.com_crdbclusters.yaml

$ kubectl create ns crdb-cluster-admin

$ curl -sL https://raw.githubusercontent.com/cockroachdb/cockroach-operator/master/manifests/operator.yaml | sed 's/namespace: default/namespace: crdb-cluster-admin/g' | kubectl -n crdb-cluster-admin apply -f -

$ kubectl create ns crdb-cluster-01

$ cat <<EOF | kubectl -n crdb-cluster-01 apply -f -
---
apiVersion: crdb.cockroachlabs.com/v1alpha1
kind: CrdbCluster
metadata:
  name: mydb
spec:
  dataStore:
    pvc:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: "5Gi"
        volumeMode: Filesystem
  tlsEnabled: true
  image:
    name: cockroachdb/cockroach:v20.2.0
  nodes: 3
EOF

[chrislovecnm]

@protosam we have not tested this, and currently it is not supported

[protosam]

From testing, I agree that it is pretty clear that this operator is designed to be deployed on a per-namespace basis right now.

When I look at the README.md I believe this turns into a question of if the operator should handle multiple namespaces?

I think it should. This seems to be an expectation of other operators, like TiDB does and so does the OLM CouchDB operator.

[jmccormick2001]

this model would be very useful, I'd like to have a single cockroach operator watch all namespaces. Today I need to support that model and do so by creating the cockroach resources outside of the operator.

[chrislovecnm]

We have that product requirement backloged