provide a way to add custom waiting functionality for resource types (or specific resources)

[cppforlife]

to support advanced waiting behaviour, possible implementations:

• support an annotation that includes a starlark snippet (may be applies to specific resource vs all resource kinds); starlark code will have access to resource and associated resources to check on done-ness

• support an annotation that indicates resource is done and successful; annotation could be populated by controllers running in a cluster

[cppforlife]

related: https://github.com/k14s/kapp/issues/26

[cppforlife]

metadata:
  name: foo
  annotations:
    kapp.k14s.io/reconcile-state: "ok|fail|ongoing"
    kapp.k14s.io/reconcile-info: "somethig happened"

• mutating webhook intercepts all requests
• mutating webhook adds kapp.k14s.io/reconcile-state=ongoing
  • if resource matches particular pattern
    • for example, kapp.k14s.io/apps-v1-deployment-wait-minimum-replicas-available is present
• controller will watch all resources
  • ignore non-annotated ones
  • update annotations with latest status based on resource status
• kapp looks at annotations to determine whether resource is reconciled

[scottd018]

Just poking in here after our conversation yesterday, honestly, the more I think about it, as a consumer of kapp I wouldn't mind it being aligned to kubectl using something like jsonpath to accomplish what I want:

1. As a consumer of kubectl, I already have an idea about jsonpath and its capabilities
2. jsonpath is fairly well documented and easy to get information about and pull examples to get it to do what I want

Something like this for a waitRule (pardon the pseudo-terminology/pseuo-wait rule) wouldn't be bad IMO:

  - apiVersion: "my.custom.resource/v1alpha1"
    kind: "MyCustomResource"
    waitFor:
      # the below example overrides the above and shows a wait for status.created to evaluate to True
      # status:
      #   created: "true"
      - fieldQuery: "$.status.created"
        fieldValue: "true"

Or something more complex (this would be using status.conditions that is already defaulted, but gives you an idea of what a more complex example may look like):

  - apiVersion: "my.custom.resource/v1alpha1"
    kind: "MyCustomResource"
    waitFor:
      # the below example shows a wait for status.conditions.type of ready to have a status of true
      - fieldQuery: "$.status.conditions[?(@.type=='Ready')].status"  # this returns an array/slice
        fieldValue: "True"                                            # this is a string value

Obviously the above examples are fairly simplistic, but would be flexible enough to allow for users to define exactly how they would want to wait for their resources, without a ton of overhead.

[pivotaljohn]

Added "discussion" label as we're still actively hashing it out.

[aaronshurley]

Moving this issue to the Unprioritized Backlog until we have a sufficient definition of the intended solution.

[praveenrewar]

Related to #442. Can be closed with #475 (using ytt wait rules). Available in v0.48.0+.