Reloader icon indicating copy to clipboard operation
Reloader copied to clipboard

[BUG] No debug messages are logged

Open macrokernel opened this issue 5 months ago • 7 comments

Describe the bug I see no debug messages in Reloader pod log:

$ kubectl -n reloader logs -f reloader-reloader-685d49466b-wl7gs
time="2025-07-18T08:29:33Z" level=info msg="Environment: Kubernetes"
time="2025-07-18T08:29:33Z" level=info msg="Starting Reloader"
time="2025-07-18T08:29:33Z" level=warning msg="KUBERNETES_NAMESPACE is unset, will detect changes in all namespaces."
time="2025-07-18T08:29:33Z" level=warning msg="namespace-selector is set, will only detect changes in namespaces with these labels: reloader.stakater.com/match_namespace."
time="2025-07-18T08:29:33Z" level=info msg="created controller for: configMaps"
time="2025-07-18T08:29:33Z" level=info msg="Starting Controller to watch resource type: configMaps"
time="2025-07-18T08:29:33Z" level=info msg="created controller for: secrets"
time="2025-07-18T08:29:33Z" level=info msg="Starting Controller to watch resource type: secrets"
time="2025-07-18T08:29:33Z" level=info msg="created controller for: namespaces"
time="2025-07-18T08:29:33Z" level=info msg="Starting Controller to watch resource type: namespaces"
time="2025-07-18T08:29:33Z" level=info msg="added namespace to be watched: opensearch"
time="2025-07-18T09:03:25Z" level=info msg="Changes detected in 'opensearch-dashboards-cert' of type 'SECRET' in namespace 'opensearch'; updated 'opensearch-dashboards' of type 'Deployment' in namespace 'opensearch'"
time="2025-07-18T09:03:29Z" level=info msg="Changes detected in 'opensearch-transport-cert' of type 'SECRET' in namespace 'opensearch'; updated 'opensearch-nodes' of type 'StatefulSet' in namespace 'opensearch'"

despite of logLevel being set to debug in Reloader deployment:

  Containers:
   reloader-reloader:
    Image:      ghcr.io/stakater/reloader:v1.4.5
    Port:       9090/TCP
    Host Port:  0/TCP
    Args:
      --log-level=debug
      --namespace-selector=reloader.stakater.com/match_namespace
      --reload-on-create=true
      --reload-strategy=annotations

To Reproduce Reloader helm values:

reloader:
  logLevel: debug
  namespaceSelector: reloader.stakater.com/match_namespace
  readOnlyRootFileSystem: true
  reloadOnCreate: true
  reloadStrategy: annotations
  watchGlobally: true

Expected behavior More details in Reloader pod logs. Perhaps Reloader simply does not log any debug messages?

Environment

  • Reloader Helm chart Version: 2.1.5
  • Kubernetes Version: v1.32.3+k3s1

macrokernel avatar Jul 18 '25 10:07 macrokernel

Hi, I've faced the same issue, e.g. no debug messages with reloader v 1.10 and kubernetes v1.31.6:

time="2025-07-21T13:54:25Z" level=info msg="Environment: Kubernetes"
time="2025-07-21T13:54:25Z" level=info msg="Starting Reloader"
time="2025-07-21T13:54:25Z" level=warning msg="KUBERNETES_NAMESPACE is unset, will detect changes in all namespaces."
time="2025-07-21T13:54:25Z" level=info msg="created controller for: configMaps"
time="2025-07-21T13:54:25Z" level=info msg="Starting Controller to watch resource type: configMaps"
time="2025-07-21T13:54:26Z" level=info msg="created controller for: secrets"
time="2025-07-21T13:54:26Z" level=info msg="Starting Controller to watch resource type: secrets"

Evalle avatar Jul 21 '25 14:07 Evalle

@macrokernel what information would you like to see on the debug level? And can you please tell more about the environment? Like how many namespaces? How many deployments? etc. and what issue are you trying to resolve?

rasheedamir avatar Jul 22 '25 16:07 rasheedamir

Hey @rasheedamir thanks for looking into it. I'm trying to understand why reloader is eating so much memory on one of my clusters. Here is some data:

Cluster Number of Secrets Number of Configmaps Number of Pods Number of Deployments Number Of Namespaces
A 1736 1050 8944 520 299
B 1805 796 1937 550 216

On cluster A the memory consumption of reloader is around 80 MB:

Image On cluster B it is 16GB: Image

So ideally in the debug messages I would like to see what is going on there and why so much difference. Thx.

Evalle avatar Jul 23 '25 08:07 Evalle

Hi @rasheedamir, I would like to see the actual restart strategy (update env, update annotations, restart) used in each update.

macrokernel avatar Jul 23 '25 13:07 macrokernel

@Evalle thanks for sharing details; is it same version of Reloader on both clusters and in general both clusters have similar specs?

@macrokernel Reloader uses the restart strategy you have setup on the workload yourself; if you can explain bit more with some examples what you would like to achieve then we can look into it

rasheedamir avatar Jul 23 '25 13:07 rasheedamir

I am seeing exactly the same behavior with "restart" strategy as if Reloader was configured for "reload" strategy, therefore I would like to make sure that Reloader is actually using the right strategy. There could be a mistake in my configuration or a bug in Reloader. It would be nice if Reloader could log the applied strategy in every log message about the restarts it makes.

macrokernel avatar Jul 23 '25 13:07 macrokernel

@rasheedamir I don't want to abuse this issue so I created a separate one for the memory consumption: https://github.com/stakater/Reloader/issues/977

Evalle avatar Jul 23 '25 14:07 Evalle