client-go icon indicating copy to clipboard operation
client-go copied to clipboard

Configure log levels or use custom logger

Open Vardominator opened this issue 1 year ago • 3 comments

Is it possible to configure the log level or use a custom logger for github.com/tikv/client-go/v2/txnkv? For data-intensive workloads, it is very noisy and difficult to track.

I'm seeing many INFO and WARN logs that are not actionable and clouding useful logs that I actually need to monitor:

...
[INFO] [lock_resolver.go:1081] ["resolveLock rollback"]
...
[INFO] [2pc.go:677] ["[BIG_TXN]"]
...

Vardominator avatar Dec 29 '24 16:12 Vardominator

they're also on stdout, making it impossible to build cli utils that use tikv client

aep avatar Feb 05 '25 18:02 aep

the logger is a global and wraps zap in a weird inflexible way, but there's a trick:

this is what i do now as a workaround:


import (
	pingcaplog "github.com/pingcap/log"
	"go.uber.org/zap"
)

func init() {
	l, p, _ := pingcaplog.InitLogger(&pingcaplog.Config{})
 
        // p is some sort of wrapper that doesnt do anything but is required for ReplaceGlobals later
        // l is zap, but its messed up, just throw it away and make a new one

	config := zap.NewProductionConfig()
	config.OutputPaths = []string{"stderr"}
	config.ErrorOutputPaths = []string{"stderr"}
	l, _ = config.Build()

	pingcaplog.ReplaceGlobals(l, p)
}

aep avatar Feb 05 '25 19:02 aep

It allows customizing the logger level. The logger in the logutil package corresponds to the global Logger in the github.com/pingcap/log package. You can initialize a custom-level logger and replace it。 https://github.com/pingcap/log/blob/master/log.go func init() { conf := &Config{Level: "info", File: FileLogConfig{}} logger, props, _ := InitLogger(conf) ReplaceGlobals(logger, props) } // ReplaceGlobals replaces the global Logger and SugaredLogger, and returns a // function to restore the original values. It's safe for concurrent use. ` func ReplaceGlobals(logger *zap.Logger, props *ZapProperties) func() {

globalMu.Lock()
prevLogger := globalLogger.Load()
prevProps := globalProperties.Load()
globalLogger.Store(logger)
subGlobalLogger.Store(logger.WithOptions(zap.AddCallerSkip(1)))
globalSugarLogger.Store(logger.Sugar())
globalProperties.Store(props)
globalMu.Unlock()

if prevLogger == nil || prevProps == nil {
	// When `ReplaceGlobals` is called first time, atomic.Value is empty.
	return func() {}
}
return func() {
	ReplaceGlobals(prevLogger.(*zap.Logger), prevProps.(*ZapProperties))
}

} `

linzai99 avatar Jun 17 '25 03:06 linzai99