feat(richlogger): Enhance Rich Logger Performance with Field Processing Caching
This PR introduces a performance optimization for the rich logger by implementing a caching mechanism that avoids repeated processFieldValue calls for the same field names. The solution adds a unique logger ID system and caches field processors to eliminate reflection overhead.
Changes Made
-
Logger ID System Added atomic counter loggerIDCounter to generate unique IDs for each logger instance Modified rich logger methods (WithCallerSkip, WithContext, WithDuration, WithFields) to generate new unique IDs using atomic.AddUint64(&loggerIDCounter, 1)
-
Caching Mechanism Cache Key: loggerID (e.g., "123") Cache Value: a slice of field processor functions that handle specific field types without reflection
-
Performance Optimization Before: Every field value processed through processFieldValue with reflection overhead After: Field processors cached per field name, processing happens only once per field type Benefits: Eliminates reflection overhead for repeated field names, significant performance improvement for loggers with many fields
-
Writer Interface Changes This is a break change
- Updated output function signature to accept loggerID uint64 parameter
- Modified all writer methods to pass the logger ID through the call chain
- Maintained backward compatibility with existing logging patterns
close #5067
I also create benchmark with 10 fields and 1000 fields
BenchmarkWithCache10Fields-12 353655 3351 ns/op 4103 B/op 49 allocs/op
BenchmarkWithoutCache10Fields-12 328093 3471 ns/op 4376 B/op 61 allocs/op
BenchmarkWithCache1000Fields-12 5157 229914 ns/op 196973 B/op 2029 allocs/op
BenchmarkWithoutCache1000Fields-12 4833 246770 ns/op 221026 B/op 3031 allocs/op
The benchmark shows higher performance and less memory use and allocs
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:loudspeaker: Thoughts on this report? Let us know!
i have tried many other ways, such as adding a member to the struct richlogger to avoid using the map cache, but it requires a much bigger change ...