Add a handy matcher that allows to check for just key existence

[voor]

Describe the problem/challenge you have I want to create an overlay that identifies an array entry by its key and appends to that array entry. This occurs in kapp-controller with the following:

apiVersion: kappctrl.k14s.io/v1alpha1
kind: App
metadata:
  name: simple-app
  namespace: default
spec:
  serviceAccountName: default-ns-sa
  fetch:
 # I want to append "subPath: config-step-2-template" to the end of the git option.
  - git:
      url: https://github.com/vmware-tanzu/carvel-simple-app-on-kubernetes
      ref: origin/develop
  template:
  - ytt: {}
  deploy:
  - kapp: {}

This is currently achieved with the following:

#@overlay/match by=lambda _,l,r: "git" in l

Full example in the ytt playground: https://carvel.dev/ytt/#gist:https://gist.github.com/voor/02c1d2e2d3c08fe42fd2df7659bc386a

Describe the solution you'd like I think this is a use case that might come up often enough to justify looking at either adding this to a FAQ or creating a corresponding matcher.

Anything else you would like to add: I never would have figured this out without phoning a friend.

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible" 👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[gcheadle-vmware]

Thank you @voor for creating this issue. I think a new matcher is a great suggestion, and it feels similar to the overlay.map_key() matcher function, but with only the matching of key names. It would be great to work out a name for the matcher in this issue, maybe something like overlay.map_key_name(), or overlay.map_key_exists(), up for discussion and suggestions around these names.

Adding the work around information to the FAQ seems like a valuable lightweight action that I can push through soon.

In the meantime, we could clarify some of the behavior for the new matching function. Is it sufficient for the new function to just be syntactic sugar for #@overlay/match by=lambda _,l,r: "PARAM" in l?

[voor]

In the meantime, we could clarify some of the behavior for the new matching function. Is it sufficient for the new function to just be syntactic sugar for #@overlay/match by=lambda _,l,r: "PARAM" in l?

The workaround worked great for my use case (which coincidentally was for another Carvel use case), and this was the first time I'd encountered this scenario.

Either of the suggestions for names (key_name or key_exists) seems great, would love to hear from other folks in the community which name makes more sense to newer users.

Thank you for the speedy response time, I appreciate you and your personalized responses every time I bring up an issue.

[gcheadle-vmware]

Adding the work around information to the FAQ seems like a valuable lightweight action that I can push through soon.

Following up on the ytt FAQ, example included here.

[pivotaljohn]

Given how common this use case is, including a matcher function for it — out of the box — seems appropriate.

That said, there are two parts to the solution which starts to read more like a recipe/idiom than "just" a matcher. The second part is marking the newly added key as missing_ok=True or when=0, depending on the desired behavior.

It makes me start to think this has more to do with a (remaining) gap in documentation. The core maintainer team is considering a track of work that involves authoring a number of guides. Currently, ytt lacks much "how to" style documentation.

[pivotaljohn]

Priority rationale:

As described seems to provide a lot of value. We'd support a contribution like this. 👍🏻

If someone wants to explore a more generalized matcher (e.g. akin to subset() but only matches on the structure of leaves, rather than their values), that is also interesting.

[cppforlife]

i would be interested in seeing overlay.subset() matcher extended to be able to use other matchers within it: e.g. overlay.subset({"key": overlay.any(), "foo": lambda ...})

[jorgemoralespou]

I would favor this syntax:

#@overlay/match by=key("git")

[jtigger]

Another example: https://kubernetes.slack.com/archives/CH8KCCKA5/p1632776327452000

#@overlay/match by=lambda _, left, right: left["metadata"]["name"].startswith("antrea-config-") and left["metadata"]["namespace"] == "kube-system" and left["kind"] == "ConfigMap"
---

vs.

#@overlay/match by=overlay.subset({"kind": "ConfigMap", "metadata", {"name": lambda v: v.startswith("antrea-config-")}, {"namespace": "kube-system"})