Support toggle behaviour for tasks with kind 'KudoOperator'

[akirillov]

What would you like to be added: It would be beneficial to enable Toggle behavior for tasks with kind KudoOperator to support optional dependencies and/or to allow a better install-time configuration of complex modular operators.

Here's an example using when as an additional field for passing the parameter name which controls whether to install the dependency or not:

- name: dependency
  kind: KudoOperator
  spec:
    package: "./packages/dependency"
    parameterFile: dependency-params.yaml
    when: installDependencyFlagName

The when flag mimics the behavior of the parameter property from the Toggle task:

  - name: dependency
    kind: Toggle
    spec:
      parameter: installDependencyFlagName
      resources:
        - install-dependency-job.yaml

The control parameter should not necessarily be named when, providing it just as an example.

Why is this needed: Currently, KudoOperator tasks don't support toggle behavior which significantly complicates the handling of the optional dependencies. A good example could be Spark and History Server or Kafka and Schema Registry. The only possible options to handle it today are either include the templates as-is into the parent operator and control the contents at the template level or to implement a job (via Toggle task) which deploys/removes the dependency based on a parameter value. Also, cascading updates/upgrades become complicated in these scenarios.

[asekretenko]

Note (based on additional clarifications from @akirillov): ensuring that the dependency is not installed when the flag is false (by uninstalling the dependent instance if present) is an important case. Operator developers would like to be able to rely on such behavior (which, as for now, has no good equivalent).