kube-state-metrics Set logging output to json format

What would you like to be added:

A possibility to set the format of the log output to json.

Why is this needed:

Common log aggregator and analyser uses the json format process logs.

Describe the solution you'd like

Start parameter like in prometheus --log.format=json or similar.

Additional context

May 20 '21 07:05 samuel-enderwitz-eficode

I absolutely believe that Kubernetes is doing the same thing,so it might be worth tracking those:

https://github.com/kubernetes/kubernetes/issues/104097
https://github.com/kubernetes/enhancements/issues/1602
https://github.com/kubernetes/kubernetes/issues/99270

Aug 07 '21 13:08 liangyuanpeng

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Nov 05 '21 14:11 k8s-triage-robot

/remove-lifecycle stale

Nov 08 '21 09:11 fpetkovski

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

Feb 06 '22 09:02 k8s-triage-robot

/remove-lifecycle stale

Feb 26 '22 13:02 liangyuanpeng

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

May 27 '22 13:05 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

Jun 26 '22 14:06 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue or PR with /reopen
Mark this issue or PR as fresh with /remove-lifecycle rotten
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Jul 26 '22 14:07 k8s-triage-robot

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied

After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied

After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Reopen this issue or PR with /reopen

Mark this issue or PR as fresh with /remove-lifecycle rotten

Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Jul 26 '22 14:07 k8s-ci-robot

I dont believe this has been addressed yet. Certainly i dont see this being documented. https://github.com/kubernetes/kube-state-metrics/blob/master/docs/cli-arguments.md

/reopen /remove-lifecycle rotten

Aug 02 '22 09:08 dmpe

@dmpe: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

I dont believe this has been addressed yet. Certainly i dont see this being documented. https://github.com/kubernetes/kube-state-metrics/blob/master/docs/cli-arguments.md

/reopen /remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Aug 02 '22 09:08 k8s-ci-robot

@fpetkovski @liangyuanpeng can you please reopen this and mark this somehow so that it wont be closed next time :) until being addressed?

Thanks.

Aug 02 '22 09:08 dmpe

/lifecycle frozen

Aug 02 '22 09:08 mrueg

This can be done by switching to structured logging. More info about the migration can be found in https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/migration-to-structured-logging.md.

/help

Aug 03 '22 09:08 dgrisonnet

@dgrisonnet: This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

Why are we solving this issue?
To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
Does this issue have zero to low barrier of entry?
How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed by commenting with the /remove-help command.

In response to this:

This can be done by switching to structured logging. More info about the migration can be found in https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/migration-to-structured-logging.md.

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Aug 03 '22 09:08 k8s-ci-robot

I wonder how to approach json logging further. klog does not seem to support JSON logging (no format option), and people use other libraries (uber zap), or create workarounds (?minikube - https://github.com/kubernetes/minikube/blob/master/pkg/minikube/out/out.go#L67). What is preffered way/rules here for kube-state-metrics project (being under kubernetes github group? Thanks for the tips.

Aug 20 '22 12:08 dmpe

uber zap prototype

add below to main.go:

....
	loggingOutput := opts.Logformat
	logger, err := initLogger(loggingOutput)
	if err != nil {
		panic(err)
	}
....

also for example in main.go:

func initLogger(loggingOutput string) (*zap.Logger, error) {
	var zaplog zap.Config

	switch loggingOutput {
        // TODO: Look at more options and how they work
	case "json":
		zaplog = zap.NewProductionConfig() // json format
		zaplog.DisableCaller = true
		zaplog.Sampling = nil
	default:
		zaplog = zap.NewDevelopmentConfig() // console format
	}

	logger, err := zaplog.Build()
	if err != nil {
		return nil, err
	}
	defer logger.Sync()

	return logger, nil
}

in options.go

o.flags.StringVar(&o.Logformat, "logformat", "logfmt", "Use logfmt or json logging.")

example result:

./kube-state-metrics --logformat json
{"level":"info","ts":1661599072.9320667,"msg":"Testing JSON logging"}
I0827 13:17:52.932231   39737 server.go:92] "Used default resources"

Aug 27 '22 11:08 dmpe

for what it is worth

Prometheus uses https://github.com/go-kit/log; the alternative to go-kit or zap could be also https://github.com/rs/zerolog

Aug 27 '22 11:08 dmpe

I am also facing the same problem. Any updates on when it will be implemented?

Aug 30 '22 21:08 AshokMishra

kube-state-metrics kube-state-metrics copied to clipboard

Set logging output to json format

Guidelines

kube-state-metrics
kube-state-metrics copied to clipboard