kubeclient
kubeclient copied to clipboard
Informer.watch is not guaranteed to receive all changes during restart
Using the Infomer.watch method that yields notices from the watch endpoint is not guaranteed to receive all events in case the informer restarts the watcher:
- Informer fills/replaces cache from
getrequest and stores the resource version - Informer starts
watchrequest with the resource version from the get - Watch request stops (timeout expires, connection drops, ...)
- Some changes happen in Kubernetes cluster before the next get request (those will not be yielded to the
watchblock) - Repeat 1: the get will receive the state with the changes happened in 4 and update list
- Repeat 2: but the watch now will start after those changes, so
watchwill not be yielded with the changes happened in 4.
Potential solutions:
- Document the behaviour of that
watchinterface can miss changes (and thatlistis the only source of truth) - Only request
getonce wheninformer.start_workeris called, when watch request stops, only restart the watch with the latest seen resource version