argocd-server-tls continuously reloaded

[dubainerd]

Checklist:

• [*] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [*] I've included steps to reproduce the bug.
• [*] I've pasted the output of argocd version.

Describe the bug

When the argocd-server-tls secret is present, the server continuously reloads it. This gives issues in web interface responsiveness

NB: the certificate has a TTL of 90 days, it's not being renewed in the timeframe of the logs below

To Reproduce

In a cluster where cert-manager is configured (you need an issuer in the argo-cd namespace), enable the server ingress and configure it to autogenerate the certificate (snippet of values below, installed using chart v6.2.1)

configs:
  params:
    server.insecure: true

server:
  ingress:
    enabled: true
    ingressClassName: nginx
    annotations:
      nginx.ingress.kubernetes.io/backend-protocol: HTTP
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      cert-manager.io/issuer: "YOUR_ISSUER_S_NAME"
    tls: true

Expected behavior

The certificate should be reloaded only when its serial changes

Screenshots

Version

argocd: v2.10.1+a79e0ea
  BuildDate: 2024-02-14T17:37:43Z
  GitCommit: a79e0eaca415461dc36615470cecc25d6d38cefb
  GitTreeState: clean
  GoVersion: go1.21.3
  Compiler: gc
  Platform: linux/amd64

Logs

UTC+4

time="2024-02-22T07:13:29Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls"
time="2024-02-22T07:13:39Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls"
time="2024-02-22T07:13:49Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls" 
time="2024-02-22T07:13:59Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls" 
time="2024-02-22T07:14:00Z" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=WatchResourceTree grpc.service=application.ApplicationService grpc.start_time="2024-02-22T07:11:06Z" grpc.time_ms=174681.06 span.kind=server system=grpc 
time="2024-02-22T07:14:00Z" level=info msg="finished streaming call with code OK" grpc.code=OK grpc.method=Watch grpc.service=application.ApplicationService grpc.start_time="2024-02-22T07:11:05Z" grpc.time_ms=175039.48 span.kind=server system=grpc
time="2024-02-22T07:14:09Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls"
--
Thu, Feb 22 2024 11:14:19 am | time="2024-02-22T07:14:19Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls"
Thu, Feb 22 2024 11:14:29 am | time="2024-02-22T07:14:29Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls"
Thu, Feb 22 2024 11:14:39 am | time="2024-02-22T07:14:39Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls"
Thu, Feb 22 2024 11:14:49 am | time="2024-02-22T07:14:49Z" level=info msg="Loading TLS configuration from secret argo-cd/argocd-server-tls"
Thu, Feb 22 2024 11:14:51 am | time="2024

[drmaciej]

Same happening here, seeing lots of those log messages, on 2.10.2. Appears related to https://github.com/argoproj/argo-cd/issues/11915.

We also observed lots of errors in the UI, on 2.8.4, just like in https://github.com/argoproj/argo-cd/issues/15807. Have not seen them yet in 2.10.2, so not sure if the errors in the UI are related to the TLS config reload.

[diranged]

We are seeing this behavior as well...

[lchastel]

Hello

I have the same issue. Lot's of next message in the log. {"level":"info", "msg":"Loading TLS configuration from secret argocd/argocd-server-tls"}

The message appears around 1000 times by hour.

What is the usage of this message ? is it needed when using argocd in production ?

Would it be possible to change the log level to debug ? So the message will only appear if we are using argocd in debug mode.

argocd-server: v2.10.6+d504d2b BuildDate: 2024-04-05T00:27:47Z GitCommit: d504d2b1d92f0cf831a124a5fd1a96ee29fa7679 GitTreeState: clean GoVersion: go1.21.3 Compiler: gc Platform: linux/amd64 Kustomize Version: v5.2.1 2023-10-19T20:13:51Z Helm Version: v3.14.3+gf03cc04 Kubectl Version: v0.26.11 Jsonnet Version: v0.20.0

[eduartua]

This might be related to https://github.com/argoproj/argo-cd/pull/14522

[rwojsznis]

anyone still encountering this behavior? on 2.11.3 and still seeing tons of those level info messages 🤔

[michaelajr]

Yeah - this is still happening. I'm on 2.10.9.

[michaelajr]

It's a hardcoded info log https://github.com/argoproj/argo-cd/blob/f358e8ddbab3af0de50ee18584b38f588a71ed2b/util/settings/settings.go#L1540