shell-operator icon indicating copy to clipboard operation
shell-operator copied to clipboard

Hooks incorrectly watch different apiVersions with the same Kind

Open wrasdf opened this issue 4 years ago • 1 comments

context

We have two different apiVersions for our aurora dababase crds:

  • aurora.rds.cloud/v1beta1
  • aurora.rds.cloud/v1beta2

And have two startup.sh for both v1beta1 & v1beta2, but in different folder.

Screen Shot 2021-07-05 at 2 08 03 pm

Take v1beta2 of standup.sh details as below:

#!/bin/bash

set -eEuo pipefail

source /app/lib/functions.sh

function __config__() {
  cat << EOF
    configVersion: v1
    onStartup: 5
    kubernetes:
    - name: startup::cluster
      apiVersion: aurora.rds.cloud/v1beta2
      kind: cluster
      keepFullObjectsInMemory: false
      allowFailure: true
      executeHookOnEvent: []
      executeHookOnSynchronization: true
      jqFilter: |
        {
          kind: .kind,
          apiVersion: .apiVersion,
          name: .metadata.name,
          namespace: .metadata.namespace,
          spec: .spec
        }
EOF
}

function __main__() {

  type="$(context::jq -r '.type')"
  context::jq -r '.' > /tmp/cluster-v1beta2-onStartup.out
  # synchronisation
  for i in $(seq 0 "$(context::jq -r '(.objects | length) - 1')"); do
    obj="$(context::jq -r '.objects['"$i"'].filterResult')"
    echo "startup::${type}"
    ......
  done

}

hook::run "$@"

Expected behavior (what you expected to happen):

As we have to support 2 versions for our operator, we expected:

  • With the same Kind
    • v1beta1 hooks could only watch the apiVersion of aurora.rds.cloud/v1beta1.
    • v1beta2 hooks could only focus on apiVersion of aurora.rds.cloud/v1beta2

Actual behavior (what actually happened):

  • With the same Kind
    • v1beta1 hooks watch both v1beta1 & v1beta2 resource
    • v1beta2 hooks watch both v1beta1 & v1beta2 resource

eg:

  • the apiVersion of delete-postgres-v1beta2 -> aurora.rds.cloud/v1beta2

Screen Shot 2021-07-05 at 2 12 07 pm

Error outputs as below:

bash-5.0# cat /tmp/cluster-v1beta1-addedModified.out
{
  "binding": "modified::cluster",
  "filterResult": {
    "apiVersion": "aurora.rds.cloud/v1beta1",
    "kind": "cluster",
    "name": "delete-postgres-v1beta2",
    "namespace": "pe",
    "spec": {
      "database_name": "postgres",
      "deletion_protection": false,
      "engine_mode": "provisioned",
      "engine_version": 12.6,
      "instance_class": "db.r5.large",
      "postgres": true,
      "primary_instance_count": 1,
      "skip_final_snapshot": true
    }
  },
  "type": "Event",
  "watchEvent": "Added"
}

Correct outputs as below:

bash-5.0# cat /tmp/cluster-v1beta2-addedModified.out
{
  "binding": "modified::cluster",
  "filterResult": {
    "apiVersion": "aurora.rds.cloud/v1beta2",
    "kind": "cluster",
    "name": "delete-postgres-v1beta2",
    "namespace": "pe",
    "spec": {
      "database_name": "postgres",
      "deletion_protection": false,
      "engine_mode": "provisioned",
      "engine_version": 12.6,
      "instance_class": "db.r5.large",
      "postgres": true,
      "primary_instance_count": 1,
      "skip_final_snapshot": true
    }
  },
  "type": "Event",
  "watchEvent": "Added"
}

If the hooks config with the same Kind but different apiVersion:

  • context::jq -r '.' could return the incorrect apiVersion as above ☝️

Environment:

  • Shell-operator version: v1.0.1
  • Kubernetes version: 1.19.6
  • Installation type (kubectl apply, helm chart, etc.): kubectl apply

wrasdf avatar Jul 05 '21 04:07 wrasdf

Hello! The behavior you've described is correct. In short, apiVersion field is not a filter but more like a hint for Kubernetes API-server.

It's not an issue inside the shell-operator but a result of the machinery inside Kubernetes. Here is the link that explains version handling for CRDs: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/ Also, check the documentation for conversion hooks: BINDING_CONVERSION

diafour avatar Jul 19 '21 08:07 diafour