#63 - Operator configuration

[burmanm]

What is missing? I'm opening this for discussion, but I'm not a fan of adding env variables for configuration. They're not typed and not easily validated in the deployment configuration. Nothing guarantees that the support for such env is still in place or that the env was correctly typed.

In the cass-operator I made some work to move on to kubebuilder config and I would personally chose that route rather than adding new env variables or command line parameters.

┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: K8OP-194

[jsanda]

For reference, https://book.kubebuilder.io/component-config-tutorial/tutorial.html. I agree that this is what we should do. I can work on this.

[jsanda]

@burmanm I read through the tutorial and I also looked at example_test.go. It looks like the configuration has to come from a file and can then be parsed into a custom type. This is great for providing validations, and I think I see how it can be used for #49.

We create a new CRD K8ssandraOperatorConfig for example that could include the control plane flag as well as the namespaces to watch. It would look something like this:

apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraOperatorConfig
health:
  healthProbeBindAddress: :8081
metrics:
  bindAddress: 127.0.0.1:8080
webhook:
  port: 9443
leaderElection:
  leaderElect: true
  resourceName: dcabfccc.k8ssandra.io
controlPlane: false
watchNamespaces:
- my-ns

The user can provide this in a ConfigMap that will be mounted in the operator pod. The file will then be parsed into a K8ssandraOperatorConfig object.

[jsanda]

@jdonenine let me summarize what the expectation is for this issue. Currently some operator configuration, e.g., control plane vs data plane, is configured via an env var. @burmanm has recommended using the kubebuilder approach which is describe at https://book.kubebuilder.io/component-config-tutorial/tutorial.html. I agree that we should do this. He has already done this for cass-operator. Some of this is already generated as part of the project scaffolding. The tutorial describes the changes necessary in the operator code.

In addition to configuring the operator for control plane vs data plane, this is where we could apply some global, default settings. For example, we could add a flag to enable host networking by default. We would want this when running in kind. It would eliminate having to set it for every K8ssandraCluster. We could also add properties for requeue timeouts which we lower in integration tests in order to speed up the tests. As a last example, we could configure registry overrides here.

@burmanm if you haven't started on this and have other stuff on your plate this could be a good ticket for someone else to pick up as a way to get ramped up.