go-redis icon indicating copy to clipboard operation
go-redis copied to clipboard

Noticing increase in latency when upgrading to v9.x.x

Open AlexMihov96 opened this issue 1 year ago • 3 comments

Description

I noticed an increase in latency for my service. I was using v8.8.0 go-redis lib version with elasticache engine v6.0.5 and I was getting ~2-3ms (p90).

Now I updated go redis to v9.3.0 and the elasticache engine to v7.0.7 with the same code and getting ~25ms (p90).

In both ways we're serving 100k rps.

Both redis clusters are exactly the same instance type (cache.r6g.xlarge) and 6 shards/24 nodes. I am using my .Get() inside a Pipelined(), I saw that Pipelines are not thread-safe anymore but I'm not using it concurrently.

Expected Behavior

Using the newest go redis library with v7.0.7 elasticache engine to have the same latency as when I was using v8.8.0 library with the v6.0.5 elasticache.

Steps to Reproduce

  1. Have a code like this
func Query() {
  predefinedKeys := []string{"bar", "foo", "test", "x", "y"}

  tStart := time.Now()

  ctx := context.Background()
  _, err := redisClient.Pipelined(ctx, func(pipe redis.Pipeliner) error {
     for _, key := range predefinedKeys {
        pipe.Get(ctx, key)
     }
     return nil
  })
  if err != nil {
     fmt.Println("Error querying data from Redis", err)
     return
  }

  tEnd := time.Now()
  fmt.Println("time elapsed:", tEnd.Sub(tStart))
}
  1. Use v6.x.x redis cluster with v8.x.x go redis lib
  2. Fetch the elapsed time from the code
  3. Now redirect the traffic towards v7.x.x cluster with newest go redis lib
  4. (I used go 1.20) doing the testing and all clusters have cluster mode ON
  5. Compare the query results

Notes

I did try using library v9.3.0 with the v6 elasticache, resulting in the same ~25ms (p90) latency as when using v9.3.0 library with v7 elasticache.

AlexMihov96 avatar Dec 01 '23 15:12 AlexMihov96

@AlexMihov96 I am also facing the same, what did you do to solve this? Did you revert to prev version?

rahulkushwaha12 avatar Feb 10 '24 13:02 rahulkushwaha12

We don't use elasticache and have increased latency as well

gugu avatar Apr 24 '24 10:04 gugu