kubelet
kubelet copied to clipboard
Clarity required around Logging Architecture of Kubelet.
Running Kubernetes 1.27.6
I have some performance issues regarding logrotation where I miss logs, I'm not here blaming Kubernetes, but there needs some clarification around the Log Architecture documents here Loggin Architecture or more details.
We have set containerLogMaxSize and containerLogMaxFiles which works fine and is rotating logs as per the settings. We are trying to debug an issue where we miss logs when there is large volume.
Now here is the disparity.
- Based on the linked document above and the 2 settings we are lead to believe that Kubelet is doing the rotation, while many online resources, forums, seem to be saying it's logrotate.
- Looking at the
/var/log/pods/folder indeed the logs are being rotate as per the 2 settings, but they are also gzipped. Nowhere does the document explain this and there's no settings indicated anywhere for compressing the logs. Which leads us to belive that the file is being zipped before the file agent can finish reading all the contents of the file. - If logrotate is truly used does Kubelet pass these 2 settings
containerLogMaxSizeandcontainerLogMaxFilesto logrotate somehow and if so how (We looked in/etc/logrotate.d/and/etc/logrotate.confand found no indication of a Kubenrnetes config pointing to/var/log/pods/)?