How to modify the YAML file of an action?

[Kaiwei-Lin]

I created a multi-node Kubernetes cluster using Kind and deployed OpenWhisk using Helm. My requirement is to run some actions on one worker node and other actions on another worker node. I noticed that after invoking an action, it creates a pod and assigns a worker node. How do I modify the Node Affinity in the YAML of this pod?

The following is a partial screenshot of the YAML.Use the command kubectl get pod wskowdev-invoker-00-62-guest-test -n openwhisk -o yaml

[Kaiwei-Lin]

The command kubectl edit pod <pod_name> -n openwhisk resulted in the following error. It seems that I need to modify the configuration of the pod during its creation.

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
# pods "wskowdev-invoker-00-205-guest-test" was not valid:
# * spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`, `spec.initContainers[*].image`, `spec.activeDeadlineSeconds`, `spec.tolerations` (only additions to existing tolerations) or `spec.terminationGracePeriodSeconds` (allow it to be set to 1 if it was previously negative)
#   core.PodSpec{
#       ... // 15 identical fields
#       Subdomain:         "",
#       SetHostnameAsFQDN: nil,
#       Affinity: &core.Affinity{
#               NodeAffinity: &core.NodeAffinity{
#                       RequiredDuringSchedulingIgnoredDuringExecution: &core.NodeSelector{
#                               NodeSelectorTerms: []core.NodeSelectorTerm{
#                                       {
#                                               MatchExpressions: []core.NodeSelectorRequirement{
#                                                       {
# -                                                             Key:      "openwhisk-role",
# +                                                             Key:      "kubernetes.io/hostname",
#                                                               Operator: "In",
# -                                                             Values:   []string{"invoker"},
# +                                                             Values:   []string{"kind-worker2"},
#                                                       },
#                                               },
#                                               MatchFields: nil,
#                                       },
#                               },
#                       },
#                       PreferredDuringSchedulingIgnoredDuringExecution: nil,
#               },
#               PodAffinity:     nil,
#               PodAntiAffinity: nil,
#       },
#       SchedulerName: "default-scheduler",
#       Tolerations:   {{Key: "node.kubernetes.io/not-ready", Operator: "Exists", Effect: "NoExecute", TolerationSeconds: &300}, {Key: "node.kubernetes.io/unreachable", Operator: "Exists", Effect: "NoExecute", TolerationSeconds: &300}},
#       ... // 13 identical fields
#   }

#

[Kaiwei-Lin]

I implemented my requirement by modifying the source code of the OpenWhisk invoker(base on gitTag: ef725a653ab112391f79c274d8e3dcfb915d59a3) .My solution is based on tag-based scheduling(tag-based-scheduling in OpenWhisk.).Since OpenWhisk deployed based on Kubernetes cannot modify the tags of invokers, I have fixed the tags in the source code.Modify code val tags = Seq(s"invoker${assignedInvokerId}") in #L114-L118. And How can we link the invoker with the node worker? When creating pod containers for invokers, I have introduced the parameter called "instanceId"，which represents the identifier of the invoker. The "instanceId" will be used for node affinity when creating invoker modify code .withValues(config.userPodNodeAffinity.value + s"${instanceId}") to this.

How to install: Before helm install owdev ./helm/openwhisk -n openwhisk --create-namespace -f mycluster.yaml, you should use the mycluster.yaml following:

whisk:
  ingress:
    type: NodePort
    apiHostName: localhost
    apiHostPort: 31001
    useInternally: false


nginx:
  httpsNodePort: 31001

scheduler:
  enabled: true

invoker:
  #use my image(base on openwhisk/invoker:ef725a6)
  imageName: "kaiweilin/openwhisk-invoker"
  imageTag: "1.2.0"

  # must use KCF as kind uses containerd as its container runtime
  containerFactory:
    impl: "kubernetes"
  # I create 5 invokers
  kubernetes:
    replicaCount: 5

if You have 5 nodeWorkers, As for executing bash commands:

kubectl label node kind-worker openwhisk-role=invoker0 openwhisk-role-invoker=invoker
kubectl label node kind-worker2 openwhisk-role=invoker1 openwhisk-role-invoker=invoker
kubectl label node kind-worker3 openwhisk-role=invoker2 openwhisk-role-invoker=invoker
kubectl label node kind-worker4 openwhisk-role=invoker3 openwhisk-role-invoker=invoker
kubectl label node kind-worker5 openwhisk-role=invoker4 openwhisk-role-invoker=invoker

How to use:

  If I have 5 invokers, their tags may be:
  invoker0 tag =>["invoker0"]
  invoker1 tag =>["invoker1"]
  invoker2 tag =>["invoker2"]
  invoker3 tag =>["invoker3"]
  invoker4 tag =>["invoker4"]

Assuming I have an action named "test", if I want it to be invoked on kind-worker（invoker0 only create pod in kind-worker), I need to update its annotation by wsk -i action update test test.js -a -a invoker-resources '["invoker0"]'

Finally, after invoking, this action will invoked in kind-worker.