ensure `include-meta-resources` is supported consistently across `kpt fn` commands

[droot]

[droot]

Mostly likely, no work is required. I would like @phanimarupaka to audit the fn commands to see if everything is in place and consistent.

[phanimarupaka]

@droot kpt fn render should also support include-meta-resources flag consistent with kpt fn eval. With this flag, eval is more powerful and the functionality provided imperatively should also be provided declaratively.

Use case: Consider an example where you want to set namespace and and also have the namespace value as substring of another field.

# mypkg/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: foo-deployment # kpt-set: ${ns}-deployment
  namespace: foo

# mypkg/Kptfile
apiVersion: kpt.dev/v1alpha2
kind: Kptfile
metadata:
  name: example
pipeline:
  mutators:
    - image: gcr.io/kpt-fn/apply-setters:v0.1
      configPath: setters-config.yaml
    - image: gcr.io/kpt-fn/set-namespace:v0.1
      configPath: namespace-config.yaml

# mypkg/setters-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: setters-config
data:
  ns: myspace

# mypkg/namespace-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ns-config
data:
  namespace: foo # kpt-set: ${ns}

So using eval users can just update setter value to myspace ONCE and it can update the other function configs as well

kpt fn eval -i gcr.io/kpt-fn/apply-setters:v0.1 --fn-config setters-config.yaml --include-meta-resources
kpt fn eval -i gcr.io/kpt-fn/set-namespace:v0.1 --fn-config namespace-config.yaml

# mypkg/namespace-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ns-config
data:
  namespace: myspace # kpt-set: ${ns}

# mypkg/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myspace-deployment # kpt-set: ${ns}-deployment
  namespace: myspace

This functionality is not offered by kpt fn render currently. Users must update both setters-config.yaml and namespace-config.yaml with value myspace and render the package to get the equivalent functionality of eval.

So we should support, --include-meta-resources feature for kpt fn render. But not sure if it should be offered as a flag for entire render command. We want it specified per function. So add a field to Function definition in Kptfile?

[phanimarupaka]

@droot @etefera Related to function selectors.

[bgrant0607]

kpt fn render still doesn't have --include-meta-resources? Is there another way to get that behavior? https://github.com/GoogleContainerTools/kpt/issues/1280#issuecomment-948887746

[bgrant0607]

cc @justinsb

[phanimarupaka]

@bgrant0607 Currently no. We are planning to include it as part of selectors. Can you please explain your use-case explaining why you need it ?

[bgrant0607]

What I'm doing now is copying Kptfile info (current the name) into a ConfigMap so that it can be read during kpt fn render.

[bgrant0607]

Example code that look for the package name copied into a ConfigMap and then uses it to change the namespace of the resources in the package. I'm calling this a "variant constructor" pattern.

func SetNamespace(ctx *krmfunction.Context, in []*yy.KubeObject, config map[string]string) ([]*yy.KubeObject, error) {
	setNamespace := ""
	for _, obj := range in {
		apiVersion := obj.GetAPIVersion()
		kind := obj.GetKind()
		oname := obj.GetName()
		if apiVersion == "v1" && kind == "ConfigMap" && oname == "kptfile.kpt.dev" {
			data, err := obj.GetOrCreateMap("data")
			if err != nil {
				return nil, err
			}
			setNamespace, _ = data.GetString("name")
			break
		}
	}
	if setNamespace == "" {
		return in, nil
	}
	for _, obj := range in {
		apiVersion := obj.GetAPIVersion()
		kind := obj.GetKind()
		if apiVersion == "v1" && kind == "Namespace" {
			if err := obj.SetName(setNamespace); err != nil {
				return nil, err
			}
			continue
		}
		if !obj.HasNamespace() {
			continue
		}
		if err := obj.SetNamespace(setNamespace); err != nil {
			return nil, err
		}
	}
	return in, nil
}

[bgrant0607]

I also think we may need to collect information from the environment and record it as local pseudo-resources, which kpt right now treats as meta-resources, to use as well known inputs in the function pipeline. For instance, the values we were treating as autosetters previously could be recorded in a context object.

[droot]

ref: https://github.com/GoogleContainerTools/kpt/pull/2781