kyverno icon indicating copy to clipboard operation
kyverno copied to clipboard

[Bug] LeaderElection related logs should be Debug level instead

Open Dentrax opened this issue 2 years ago • 3 comments

Kyverno Version

1.7.2

Description

While checking Kyverno logs, just noticed it prints out lots of leader election related logs:

kyverno-84ffd96fb5-fpn4p kyverno I0628 07:48:15.561563       1 round_trippers.go:553] GET https://10.233.0.1:443/apis/coordination.k8s.io/v1/namespaces/kyverno/leases/kyverno 200 OK in 3 milliseconds
kyverno-84ffd96fb5-fpn4p kyverno I0628 07:48:15.563599       1 round_trippers.go:553] GET https://10.233.0.1:443/apis/coordination.k8s.io/v1/namespaces/kyverno/leases/kyverno 200 OK in 1 milliseconds
kyverno-84ffd96fb5-fpn4p kyverno I0628 07:48:15.563740       1 status.go:31] webhookMonitor/WebhookStatusControl "msg"="annotation kyverno.io/webhookActive already set to 'true'" "name"="kyverno" "namespace"="kyverno"
kyverno-84ffd96fb5-7txh6 kyverno I0628 07:48:16.020497       1 round_trippers.go:553] GET https://10.233.0.1:443/apis/coordination.k8s.io/v1/namespaces/kyverno/leases/webhook-register 200 OK in 2 milliseconds
kyverno-84ffd96fb5-7txh6 kyverno I0628 07:48:16.025020       1 round_trippers.go:553] PUT https://10.233.0.1:443/apis/coordination.k8s.io/v1/namespaces/kyverno/leases/webhook-register 200 OK in 4 milliseconds
kyverno-84ffd96fb5-7txh6 kyverno I0628 07:48:16.025184       1 leaderelection.go:278] successfully renewed lease kyverno/webhook-register

Leader election logs prints lots of information in I level and our logging system pushes these logs to elasticsearch index storage unnecessarily.

It would be nice to set log level D (Debug) or something.

cc @developer-guy

Slack discussion

No response

Troubleshooting

  • [X] I have read and followed the documentation AND the troubleshooting guide.
  • [X] I have searched other issues in this repository and mine is not recorded.

Dentrax avatar Jun 28 '22 07:06 Dentrax

Can I work on this.

MdSahil-oss avatar Aug 25 '22 09:08 MdSahil-oss

Any update on this? It seems mentioned PR closed. @MdSahil-oss

Dentrax avatar Sep 05 '22 06:09 Dentrax

@Dentrax I closed the PR because the changes I made through the PR wasn't expected changes as @realshuting suggested, Would you like to explain this issue a little bit more so that It could be clear what's the expectation for the issue because this issue is also unclear to @realshuting. As you mentioned in the issue for changing the the level of logs related to the leaderElection but all the leaderElection logs are printing relevant message. slack conversation link

MdSahil-oss avatar Sep 05 '22 08:09 MdSahil-oss

@Dentrax this should no longer be the case in 1.8 can you confirm ?

eddycharly avatar Oct 20 '22 12:10 eddycharly

Hey @eddycharly, it seems doesn't fixed 👀 Running 1.8.0:

Screen Shot 2022-10-20 at 15 49 02

Dentrax avatar Oct 20 '22 12:10 Dentrax

Is this 1.8.0 ? webhook-register does not exist anymore.

eddycharly avatar Oct 20 '22 12:10 eddycharly

Yes, actually:

$ helm list
NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
kyverno kyverno         2               2022-10-20 15:40:37.344646 +0300 +03    deployed        kyverno-2.6.0   v1.8.0

Dentrax avatar Oct 20 '22 14:10 Dentrax

It would be nice to set log level D (Debug) or something.

@Dentrax this does not exist (only I or E are available). We are switching to structured logging, will be ready in 1.8.2 hopefully.

What -v level are you running ? Some logs here are not controlled by us but come from a k8s lib. Are you running with -v >= 5 ?

eddycharly avatar Oct 20 '22 14:10 eddycharly

Are you running with -v >= 5?

I was passed -v=5, is there another flag to set log verbosity?

Dentrax avatar Oct 20 '22 14:10 Dentrax

Nope but with -v=5 there’s nothing we can do, it’s hard coded in k8s lib.

eddycharly avatar Oct 20 '22 15:10 eddycharly

@Dentrax can we close this as there's nothing much we can do when -v=5 ?

eddycharly avatar Oct 20 '22 15:10 eddycharly

Oh I see. I thought I had to pass a flag to make verbosity level to info. I couldn't see any info log from kyverno containers. I did helm install with default values. If this was the expected behavior, so LGTM! We can close this one. 🤞

Dentrax avatar Oct 20 '22 15:10 Dentrax

kyverno-84ffd96fb5-7txh6 kyverno I0628 07:48:16.025184 1 leaderelection.go:278] successfully renewed lease kyverno/webhook-register

~~This line comes from kyverno itself for example.~~

Actually no.

eddycharly avatar Oct 20 '22 15:10 eddycharly

There's no reason kyverno should not output logs. Can you check on the pods ?

eddycharly avatar Oct 20 '22 15:10 eddycharly

@Dentrax ?

chipzoller avatar Feb 26 '23 14:02 chipzoller