Bug: kubeless doesn't update resources associated with custom resource definitions

[tsurankov]

BUG REPORT:

What happened:

If a HTTP Trigger on a CronJob trigger has been created, update of the trigger's spec doesn't affect the actual resource (Ingress or CronJob).

I tried to update triggers using kubeless-cli and kubectl manually. The result is the same: trigger's configuration has changed but the corresponding resource stays untouched.

What you expected to happen:

Update of a trigger updates corresponding CronJob or Ingress configuration.

How to reproduce it:

Create a trigger:

kubeless trigger cronjob create test-function -n serverless --function test-function --schedule '0 0 * * *'

Get the configuration of an actual CronJob resource:

k get cronjob trigger-test-function -n serverless -o yaml | grep schedule:

The result as expected is: schedule: 0 0 * * * Update the schedule:

kubeless trigger cronjob update test-function -n serverless --function test-function --schedule '12 12 * * *'

Now get the config from CronJob again:

k get cronjob trigger-test-function -n serverless -o yaml | grep schedule:

The result hasn't changed: schedule: 0 0 * * * Expected result: schedule: 12 12 * * *

The same issue is related to HTTP Trigger. For instance, if the parameter spec.basic-auth-secret is changed, the configuration of an ingress won't change. The only workaround I found is to delete a trigger and then create a new one.

Environment:

• Kubernetes version: v1.14.8
• Kubeless version: v1.0.7-dirty
• Cloud provider or physical cluster: kops on AWS

[andresmgot]

Thanks @tsurankov,

I can reproduce the bug, the problem with the HTTP ingress is that the annotations are not being updated if the ingress already exists. We are just updating the Ingress spec. Check the code here:

https://github.com/kubeless/http-trigger/blob/master/pkg/utils/k8sutil.go#L336

We would need to update the annotations there as well (which is what changes if you change the auth info for example).

Regarding your issue with the CronJob trigger, I am not able to reproduce it:

▶ kubeless trigger cronjob create test --function get-java-11 --schedule '1 * * * *'
INFO[0000] Cronjob trigger test created in namespace default successfully! 

▶ kubectl get cronjob -o yaml trigger-get-java-11 | grep schedule:
  schedule: 1 * * * *

▶ kubeless trigger cronjob update test --function get-java-11 --schedule '12 12 * * *'                         
INFO[0000] Cronjob trigger test updated in namespace default successfully! 

▶ kubectl get cronjob -o yaml trigger-get-java-11 | grep schedule:
  schedule: 12 12 * * *

I am not sure if there is any other error. You can check the controller logs to get more info.

[tsurankov]

Hello @andresmgot, Thank you for a quick response. I have investigated a bit more. If I create a function from the kubeless CLI, the cronjob trigger update command works well. It looks like the issue appears if a function is created from a yaml manifest file. Here is the yaml which I used for test:

---
apiVersion: kubeless.io/v1beta1
kind: Function
metadata:
  name: test
  namespace: serverless
  label:
    created-by: kubeless
    function: test
spec:
  runtime: python3.7
  timeout: "180"
  handler: function.hello
  deps: |
    hvac==0.10.3
    botocore==1.16.15

  function-content-type: text
  function: |
    def hello(event, context):
      print(event)
      return event['data']

  deployment:
    spec:
      template:
        spec:
          initContainers:
          - resources:
              limits:
                cpu: 200m
                memory: 200Mi
              requests:
                cpu: 200m
                memory: 200Mi
          containers:
            name: "test"
            resources:
              limits:
                cpu: 100m
                memory: 100Mi
              requests:
                cpu: 100m
                memory: 100Mi

Could you try to reproduce?

kubectl create ns serverless
kubectl apply -f function.yaml
kubeless trigger cronjob create test -n serverless --function test --schedule '0 0 * * *'
kubectl get cronjob trigger-test -n serverless -o yaml | grep schedule:
kubeless trigger cronjob update test --function test-function -n serverless --schedule '2 2 * * *'
kubectl get cronjob trigger-test -n serverless -o yaml | grep schedule:

I am getting **schedule: 1 * * * ***, the cronjob hasn't changed

[andresmgot]

your containers is wrong, it should be an array and you are defining an object. In fact, when I try to reproduce your issue I am getting the following error:

▶ kubeless trigger cronjob create test -n serverless --function test --schedule '0 0 * * *'

FATA[0000] Unable to find Function test in namespace serverless. Error v1beta1.Function.Spec: v1beta1.FunctionSpec.Deployment: v1.Deployment.Spec: v1.DeploymentSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.Containers: []v1.Container: decode slice: expect [ or n, but found {, error found in #10 byte of ...|tainers":{"name":"te|..., bigger context ...|yment":{"spec":{"template":{"spec":{"containers":{"name":"test","resources":{"limits":{"cpu":"100m",|... 

My recommendation is to dump the function yaml using kubeless function deploy ... --dryrun -o yaml and then tweak that one if you need to modify something.