go-zero icon indicating copy to clipboard operation
go-zero copied to clipboard

feat(richlogger): Enhance Rich Logger Performance with Field Processing Caching

Open DengY11 opened this issue 3 months ago • 2 comments

This PR introduces a performance optimization for the rich logger by implementing a caching mechanism that avoids repeated processFieldValue calls for the same field names. The solution adds a unique logger ID system and caches field processors to eliminate reflection overhead.

Changes Made

  1. Logger ID System Added atomic counter loggerIDCounter to generate unique IDs for each logger instance Modified rich logger methods (WithCallerSkip, WithContext, WithDuration, WithFields) to generate new unique IDs using atomic.AddUint64(&loggerIDCounter, 1)

  2. Caching Mechanism Cache Key: loggerID (e.g., "123") Cache Value: a slice of field processor functions that handle specific field types without reflection

  3. Performance Optimization Before: Every field value processed through processFieldValue with reflection overhead After: Field processors cached per field name, processing happens only once per field type Benefits: Eliminates reflection overhead for repeated field names, significant performance improvement for loggers with many fields

  4. Writer Interface Changes This is a break change

  • Updated output function signature to accept loggerID uint64 parameter
  • Modified all writer methods to pass the logger ID through the call chain
  • Maintained backward compatibility with existing logging patterns

close #5067

I also create benchmark with 10 fields and 1000 fields

BenchmarkWithCache10Fields-12                     353655              3351 ns/op            4103 B/op         49 allocs/op
BenchmarkWithoutCache10Fields-12                  328093              3471 ns/op            4376 B/op         61 allocs/op
BenchmarkWithCache1000Fields-12                     5157            229914 ns/op          196973 B/op       2029 allocs/op
BenchmarkWithoutCache1000Fields-12                  4833            246770 ns/op          221026 B/op       3031 allocs/op

The benchmark shows higher performance and less memory use and allocs

DengY11 avatar Sep 23 '25 08:09 DengY11

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests.

:loudspeaker: Thoughts on this report? Let us know!

codecov[bot] avatar Sep 23 '25 08:09 codecov[bot]

i have tried many other ways, such as adding a member to the struct richlogger to avoid using the map cache, but it requires a much bigger change ...

DengY11 avatar Sep 23 '25 08:09 DengY11