lettuce icon indicating copy to clipboard operation
lettuce copied to clipboard

Error in parsing list of list response under load

Open shen-jin opened this issue 2 years ago • 1 comments

Bug Report

Current Behavior

I have a custom module command which returns hgetall for multiple hash at the same time. The response is list of list. Each 1st level list element represent a hash, each 2nd level list element contains hash keys and values leaded by hash key as first element.

When I call this command at heavy load, it happens occasionally that the ArrayOutput does not have correct list. The first half of the list is repeated in the middle, which can be seen in the logs

I have captured logs from lettuce, which can be downloaded from https://shen2080-s3.s3.eu-west-1.amazonaws.com/redis-fulllog.txt.gz

I also extracted info from the log into separated file.

The command sent by lettuce can be found at https://shen2080-s3.s3.eu-west-1.amazonaws.com/redis-command.txt You can see that it is calling module command "cg.mgetHashValues" with 40 parameters, each a hash key

The redis response logged by lettuce can be found at https://shen2080-s3.s3.eu-west-1.amazonaws.com/redis-response.txt Since my hash contains some long value, I cleaned the response and replaced long value with . The cleaned version is https://shen2080-s3.s3.eu-west-1.amazonaws.com/redis-response-cleaned.txt You can see the response is list of 40 elements, each element is in turn a list of 13 elements with first element matches the query parameter in order. The redis response looks good.

Lettuce parsed the response into this ArrayOutput https://shen2080-s3.s3.eu-west-1.amazonaws.com/redis-parsed.txt Similarly I have it cleaned for readability at https://shen2080-s3.s3.eu-west-1.amazonaws.com/redis-parsed-cleaned.txt You can see that inside the 24th sub array, it started over from the first element and then run through the whole list again til the end.

It is the same patten every time this issue happened. the list start over somewhere in the middle

Environment

  • Lettuce version(s): 6.1.4.RELEASE 6.1.5.RELEASE
  • Redis version: 6.2.4

shen-jin avatar Jan 16 '22 23:01 shen-jin