expirable-cache icon indicating copy to clipboard operation
expirable-cache copied to clipboard

Optimize core cache operations for performance, Go 1.24

Open analytically opened this issue 7 months ago • 4 comments
trafficstars

Improve performance in frequently used cache methods:

  1. Values method:

    • Move time.Now() outside of lock
    • Simplify iteration
  2. DeleteExpired method:

    • Replace key slice allocation with direct list traversal
    • Cache time.Now() call for efficiency, move out of lock
    • Avoid redundant map lookups
    • Properly handle list traversal during element removal
  3. Add method:

    • Cache time.Now() and move out of lock

All optimizations preserve API compatibility while reducing allocations and CPU work.

analytically avatar Mar 28 '25 11:03 analytically

goos: darwin
goarch: arm64
pkg: github.com/go-pkgz/expirable-cache/v3
cpu: Apple M1 Pro
                       │  master.txt  │              perf.txt               │
                       │    sec/op    │    sec/op     vs base               │
LRU_Rand_NoExpire-10     179.7n ± ∞ ¹   169.3n ± ∞ ¹   -5.79% (p=0.016 n=5)
LRU_Freq_NoExpire-10     173.0n ± ∞ ¹   167.5n ± ∞ ¹        ~ (p=0.222 n=5)
LRU_Rand_WithExpire-10   203.2n ± ∞ ¹   176.2n ± ∞ ¹  -13.29% (p=0.008 n=5)
LRU_Freq_WithExpire-10   197.2n ± ∞ ¹   175.2n ± ∞ ¹  -11.16% (p=0.008 n=5)
geomean                  187.9n         172.0n         -8.44%
¹ need >= 6 samples for confidence interval at level 0.95

analytically avatar Mar 28 '25 11:03 analytically

Moving time.Now() before acquiring the lock reduces the critical section duration, which can significantly improve concurrency in high-contention scenarios. The exact timestamp precision is not critical (a few microseconds earlier doesn't matter), and potentially you could say that the method called time is the mark. And time.Now() can be relatively expensive (vDSO/syscall), so performing it outside the lock helps reduce contention.

analytically avatar Mar 28 '25 11:03 analytically

Pull Request Test Coverage Report for Build 14173478019

Details

  • 13 of 13 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 100.0%

Totals Coverage Status
Change from base Build 13825013328: 0.0%
Covered Lines: 168
Relevant Lines: 168

💛 - Coveralls

coveralls avatar Mar 31 '25 14:03 coveralls

I've added the same optimisations to v1 and v2, re-run the benchmarks and updated the results in readme, and also reverted the upgrade of go version to 1.24 as it's up to the caller and doesn't make sense to make a requirement from the library from my perspective. I'll release the new version once it's merged.

@umputun, could you please take a look?

paskal avatar Mar 31 '25 14:03 paskal