Annotation to choose which rabbitmq-operator manages instance

[james-callahan]

Is your feature request related to a problem? Please describe. To be able to run more than one instance of the rabbitmq operator in a cluster, we'd like a way to specify which rabbitmq-operator should act on a given Cluster manifest

Describe the solution you'd like Add an environment variable to the operator, e.g. OPERATOR_NAME which is checked against a RabbitmqCluster annotation such as rabbitmq.com/cluster-operator/name.

Additional context The proposed solution is similar to how you can have multiple ingresses in a given cluster selected by kubernetes.io/ingress.class

[ChunyiLyu]

@james-callahan Thanks for using cluster operator 😸

This is an interesting idea and can certainly be done with the operator. I would like to understand what specific use case it could enable for you. With the example of multiple ingress controllers, they are functionally different and I can see why it's necessary for certain environments. For cluster operators, later versions only add features and never remove any feature support. Could you explain more your needs for such this feature request?

On the other hand, if you wish to have multiple operators in your k8s clusters each managing a set of rabbits, you can do that today with namespacing the cluster operator by setting the environment variable OPERATOR_SCOPE_NAMESPACE on the operator deployment. For like:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rabbitmq-cluster-operator
  namespace: rabbitmq-system
spec:
  template:
...
    spec:
      containers:
      - command:
        - /manager
        env:
        - name: OPERATOR_SCOPE_NAMESPACE
           value: <a-specific-namespace-for-this-operator>
        image: rabbitmqoperator/cluster-operator:1.7.0
        name: operator
...

This will make the operator only watch for rabbit clusters in one namespace and you could have multiple operators deployed in different namespaces. They will only manage rabbit clusters created in namespace <a-specific-namespace-for-this-operator>.

[github-actions[bot]]

This issue has been marked as stale due to 60 days of inactivity. Stale issues will be closed after a further 30 days of inactivity; please remove the stale label in order to prevent this occurring.

[james-callahan]

For cluster operators, later versions only add features and never remove any feature support. Could you explain more your needs for such this feature request?

e.g. testing a new version of the operator without shutting down the old operator.

[ChunyiLyu]

@james-callahan I see. With your use case I think setting the environment variable OPERATOR_SCOPE_NAMESPACE should work for you. Could you give it a go and let us know? Meanwhile, I will add documentation about this feature in rabbitmq.com.

[james-callahan]

@james-callahan I see. With your use case I think setting the environment variable OPERATOR_SCOPE_NAMESPACE should work for you. Could you give it a go and let us know? Meanwhile, I will add documentation about this feature in rabbitmq.com.

That's not sufficient: e.g.

• the (old) operator is running in cluster-wide mode
• the (new) operator runs with OPERATOR_SCOPE_NAMESPACE ==> both operators would be fighting over the resources in OPERATOR_SCOPE_NAMESPACE.

What's needed is a way to opt specific rabbitmq instances into an operator (via some field or an annotation)

[durairajasivam]

@james-callahan, I see. With your use case, I think setting the environment variable OPERATOR_SCOPE_NAMESPACE should work for you. Could you give it a go and let us know? Meanwhile, I will add documentation about this feature in rabbitmq.com.

Hi @ChunyiLyu Do you have the documentation for this?

I have tried OPERATOR_SCOPE_NAMESPACE, but it still conflicts with another operator running in a different namespace. I have even tried to apply the namespace for each service, role binding, role, deployment, etc., in operator, but it seems it does not help.

Appreciate your guidance on this, thanks

[tormig-softronic]

@james-callahan, I see. With your use case, I think setting the environment variable OPERATOR_SCOPE_NAMESPACE should work for you. Could you give it a go and let us know? Meanwhile, I will add documentation about this feature in rabbitmq.com.

Hi @ChunyiLyu Do you have the documentation for this?

I have tried OPERATOR_SCOPE_NAMESPACE, but it still conflicts with another operator running in a different namespace. I have even tried to apply the namespace for each service, role binding, role, deployment, etc., in operator, but it seems it does not help.

Appreciate your guidance on this, thanks

When I try to install this as a non-admin user i get: Error: INSTALLATION FAILED: failed to install CRD crds/crd-rabbitmq-cluster.yaml: customresourcedefinitions.apiextensions.k8s.io is forbidden: User "tortes" cannot create resource "customresourcedefinitions" in API group "apiextensions.k8s.io" at the cluster scope So I am not sure what the "OPERATOR_SCOPE_NAMESPACE" variable is supposed to do but it is clearly trying to create CRD's at cluster level and then I understand why you still only can have one running at the time.

[Zerpet]

@tormig-softronic please open a new discussion for your issue. What you are reporting is not related to this issue.

[dgshue]

I'm attempting to do the same thing here - multiple clusters in different namespaces (dev, val, prd). I'm struggling to understand what the deployments should look like for each namespace and like wise for the operator itself, any hints would be greatly appreciated

[durairajasivam]

I'm attempting to do the same thing here - multiple clusters in different namespaces (dev, val, prd). I'm struggling to understand what the deployments should look like for each namespace and like wise for the operator itself, any hints would be greatly appreciated

Hi dgshue,

1. Create the CRD and the associated Operator.
2. The Operator will run in the rabbitmq-system namespace. By default, it will oversee all your RabbitMQ clusters across different namespaces.
3. Once the rabbitmq-cluster-operator is operational, you can initiate your RabbitMQ cluster. To do this, apply the deployment using the RabbitmqCluster kind for each respective namespace.

I hope this helpsl.

[dgshue]

I was able to get this working, but I ended up with different operators in their respective namespaces scoped to their own namespace. Thanks for the help!

On Tue, Sep 26, 2023 at 6:18 PM durairajasivam @.***> wrote:

I'm attempting to do the same thing here - multiple clusters in different namespaces (dev, val, prd). I'm struggling to understand what the deployments should look like for each namespace and like wise for the operator itself, any hints would be greatly appreciated

Hi dgshue, Create CRD and Operator it will be running one namespace called rabbitmq-system, it will manage all your RabbitMQ clusters regrades of namespaces by defaults. once the rabbitmq-cluster-operator in running state you can create your rabbitmq cluster by applying deployment kind RabbitmqCluster for each namespaces. hope this helps!.

— Reply to this email directly, view it on GitHub https://github.com/rabbitmq/cluster-operator/issues/699#issuecomment-1736380410, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADC2EXHI2HPVORSUPKW36A3X4NID7ANCNFSM45IGUIVQ . You are receiving this because you commented.Message ID: @.***>