shell-operator icon indicating copy to clipboard operation
shell-operator copied to clipboard

Hook gets deleted Pod in snapshot

Open diafour opened this issue 5 years ago • 2 comments

++ jq -r .snapshots.pods
+ pods='[
  {
    "filterResult": {
      "hostIP": null,
      "initialHostIP": "192.168.xxx.yyy",
      "name": "NAME-6c97fcc5f4-mh6f5"
    }
  },
  {
    "filterResult": {
      "hostIP": "192.168.199.135",
      "initialHostIP": null,
      "name": "NAME-6c97fcc5f4-mnx2c"
    }
  }
]'

kubectl:

root@kube-master-0 / # kubectl get pods -l app=NAME
NAME                         READY   STATUS    RESTARTS   AGE
NAME-6c97fcc5f4-mnx2c   0/1     Running   0          9h

Configuration:

configVersion: v1
    kubernetes:
    - name: pods
      group: main
      keepFullObjectsInMemory: false
      executeHookOnEvent: ["Added", "Modified"]
      executeHookOnSynchronization: true
      apiVersion: v1
      kind: Pod
      namespace:
        nameSelector:
          matchNames: ["namespace-name"]
      labelSelector:
        matchLabels:
          app: "NAME"
      jqFilter: |
        {
          "name": .metadata.name,
          "hostIP": .status.hostIP,
          "initialHostIP": .metadata.annotations."NAME/initial-host-ip"
        }

The first intention is a snapshot not invalidated properly because of the logic behind the option keepFullObjectsInMemory: false.

diafour avatar Nov 18 '20 07:11 diafour

Update after some research.

There is a possibility to get nil from DeletedFinalStateUnknown.Obj: https://github.com/flant/shell-operator/blob/master/pkg/kube_events_manager/resource_informer.go#L271-L274. The best we can do here is to use same KeyFunc for index in CachedResources. For Event mode we can mimic Informers behavior and use cached filterResult and/or object with additional "stale" flag in binding context.

diafour avatar Dec 15 '20 11:12 diafour

Also, we should consider implementing custom caching for Informers to make keepFullObjectsInMemory: false really working.

diafour avatar Dec 15 '20 11:12 diafour