Add new Kind: OperatorConfig OR encapsulate its functionality

[aRustyDev]

Feature Request

Disclaimer : FNG here, very likely that I'm missing some already defined resource.

The Problem

I am trying to use Terraform to automated and document my clusters deployment. I am using a combination of Helm and OLM to try and deploy both managed and unmanaged applications. But I have some applications that need to manage external cloud resources like S3, Route53, or Certificates and require setting annotations or application settings like shown in the external-secrets-operator

Example

apiVersion: operator.external-secrets.io/v1alpha1
kind: OperatorConfig
metadata:
  name: cluster
spec:
  prometheus:
    enabled: true
    service:
      port: 8080
  resources:
   requests:
     cpu: 10m
     memory: 96Mi
   limits:
     cpu: 100m
     memory: 256Mi

Possible Solutions

1. Basically just copy the external-secrets Kind OperatorConfig.
2. Encapsulate the OperatorConfig functionality into the Kind Subscription

Example 1

apiVersion: operators.coreos.com/v1alpha1
kind: OperatorConfig
metadata:
  name: cluster
spec:
  prometheus:
    enabled: true
    service:
      port: 8080
  resources:
   requests:
     cpu: 10m
     memory: 96Mi
   limits:
     cpu: 100m
     memory: 256Mi

Example 2

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: myApp
spec:
  package: myApp
  channel: alpha
  config:
    env:
      - name: ARGS
        value: "-v=10"
    clientSpecs:                <---- Shiny
      prometheus.enabled: True  <---- New
      service.port: 8080        <---- Feature
    resources:
     requests:
       cpu: 10m
       memory: 96Mi
     limits:
       cpu: 100m
       memory: 256Mi

[aRustyDev]

This would likely involve adding a param to type SubscriptionConfig similar to NodeSelector map[string]string https://pkg.go.dev/github.com/operator-framework/[email protected]/pkg/operators/v1alpha1#SubscriptionConfig

https://github.com/operator-framework/api/blob/v0.14.0/pkg/operators/v1alpha1/subscription_types.go#L42

Updating this would likely be the first step to enabling Example 2

// SubscriptionConfig contains configuration specified for a subscription.
type SubscriptionConfig struct {
	// Selector is the label selector for pods to be configured.
	// Existing ReplicaSets whose pods are
	// selected by this will be the ones affected by this deployment.
	// It must match the pod template's labels.
	Selector *metav1.LabelSelector `json:"selector,omitempty"`

	// NodeSelector is a selector which must be true for the pod to fit on a node.
	// Selector which must match a node's labels for the pod to be scheduled on that node.
	// More info: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/
	// +optional
	NodeSelector map[string]string `json:"nodeSelector,omitempty"`

	// ClientValues is a map of values to populate values in the container.
	// +optional
	ClientValues map[string]string `json:"clientValues,omitempty"`

	// Tolerations are the pod's tolerations.
	// +optional
	Tolerations []corev1.Toleration `json:"tolerations,omitempty"`

	// Resources represents compute resources required by this container.
	// Immutable.
	// More info: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
	// +optional
	Resources *corev1.ResourceRequirements `json:"resources,omitempty"`

	// EnvFrom is a list of sources to populate environment variables in the container.
	// The keys defined within a source must be a C_IDENTIFIER. All invalid keys
	// will be reported as an event when the container is starting. When a key exists in multiple
	// sources, the value associated with the last source will take precedence.
	// Values defined by an Env with a duplicate key will take precedence.
	// Immutable.
	// +optional
	EnvFrom []corev1.EnvFromSource `json:"envFrom,omitempty"`
	// Env is a list of environment variables to set in the container.
	// Cannot be updated.
	// +patchMergeKey=name
	// +patchStrategy=merge
	// +optional
	Env []corev1.EnvVar `json:"env,omitempty" patchMergeKey:"name" patchStrategy:"merge"`

	// List of Volumes to set in the podSpec.
	// +optional
	Volumes []corev1.Volume `json:"volumes,omitempty"`

	// List of VolumeMounts to set in the container.
	// +optional
	VolumeMounts []corev1.VolumeMount `json:"volumeMounts,omitempty"`
}

[kevinrizza]

@aRustyDev I'm maybe a little bit lost on the details here. What would you expect OLM to do with those prometheus fields? The subscription config fields have a 1:1 relationship with fields in the kubernetes podspec, and OLM just has logic to hardwire those values to any deployment specified in the operator's CSV. But the prometheus fields you're describing here aren't native to any specific kubernetes resource, so I'm not sure what they would be used for.

In your example, it looks like the external secrets operator has a configuration API, and presumably they ship a controller that watches that resource and knows how to wire up that configuration to whatever resources consume it. But since that isn't generic to any arbitrary controller deployment, I'm not sure that subscription config or a generic OLM api is the right place for this. Rather, I would expect this kind of configuration to be defined by a resource that is exposed by whatever operator cares about it.

[joelanford]

This seems more like a request for a templating feature, where the bundle could expose some default values, perhaps a schema for the values, and then OLM would know how to consume admin-provide values as overrides (and possibly validate them from an provided schema) and apply them when rendering manifests from the bundle.

Is that along the lines of what you're asking for @aRustyDev?