es-operator icon indicating copy to clipboard operation
es-operator copied to clipboard

Use pod and node informers to reduce APIServer load

Open mikkeloscar opened this issue 4 years ago • 2 comments

This introduces the use of informers for listing pods and nodes. It's inspired by what is done in external-dns: https://github.com/kubernetes-sigs/external-dns/blob/805555872ee7e27ab15802a86e313120432117fa/source/ingress.go#L78

The main idea of informers are that they use an internal cache which is maintained via a watch. Whenever the operator will do a list pods or list nodes it will read from the internal cache instead of making an API call to the APIServer thus reducing the load which is usually caused by listing pods in big clusters.

This needs a bit of testing before it can be considered. I had to change a lot of []v1.Pod to []*v1.Pod because of the informer semantics, but I don't fully understand the API yet so I won't to make sure we are still using the values in a safe way. Additionally we need to understand how this watch behaves when there are connection issues to the APIServer.

mikkeloscar avatar Oct 30 '19 22:10 mikkeloscar

👍

njuettner avatar Nov 06 '19 08:11 njuettner

Is it still WIP or ready for review?

otrosien avatar Nov 17 '22 06:11 otrosien