"Redis server returned invalid response"

[daurnimator]

The response seems to be valid; but I get an error:

[error] 23868#0: *148 Redis server returned invalid response near pos 18 in "+OK
+QUEUED
*1
*85
$1
1
$1
2
$1
3
$1
4
$1
5
$1
6
$1
7
$1
8
$1
9
$2
10
$2
11
$2
12
$2
13
$2
14
$2
15
$2
16
$2
17
$2
18
$2
19
$2
20
$2
21
$2
22
$2
23
$2
24
$2
25
$2
26
$2
27
$2
28
$2
29
$2
30
$2
31
$2
32
$2
33
$2
34
$2
35
$2
36
$2
37
$2
38
$2
39
$2
40
$2
41
$2
42
$2
43
$2
44
$2
45
$2
46
$2
47
$2
48
$2
49
$2
50
$2
51
$2
52
$2
53
$2
54
$2
55
$2
56
$2
57
$2
58
$2
59
$2
60
$2
61
$2
62
$2
63
$2
64
$2
65
$2
66
$2
67
$2
68
$2
69
$2
70
$2
71
$2
72
$2
73
$2
74
$2
75
$2
76
$2
77
$2
78
$2
79
$2
80
$2
81
$2
82
$2
83
$2
84
$2
85
" while reading response header from upstream, client: XXXXXXXXX, server: , request: "GET XXXXXXXXXXXXX", subrequest: "/redis", upstream: "redis2://127.0.0.1:6379", host: "XXXXXXXXXXXXX"

[agentzh]

Hello!

On Thu, Aug 16, 2012 at 11:06 AM, daurnimator [email protected] wrote:

The response seems to be valid; but I get an error:

[error] 23868#0: *148 Redis server returned invalid response near pos 18 in "+OK +QUEUED [...]

Could you show your nginx config file (or any related external Lua code)?

Thanks! -agentzh

[daurnimator]

As a minimal example, it occurs with: (log is contains a hash with a timestamp field)

local parser = require "redis.parser"
local function redis_queries ( reqs )
    local n_reqs = #reqs
    local raw_reqs = { }
    for i = 1 , n_reqs do
        raw_reqs [ i ] = parser.build_query ( reqs [ i ] )
    end
    local res = ngx.location.capture ( "/redis?" .. n_reqs , {
            body = table.concat ( raw_reqs ) ;
        } )
    if res.status ~= 200 or not res.body then
        ngx.log(ngx.ERR, "failed to query redis")
        ngx.exit(500)
    end
    return parser.parse_replies ( res.body , n_reqs )
end

ngx.print(redis_queries {
    {"MULTI"},
    { "sort" , "log" , "BY" , "log:*->timestamp" } ;
    {"EXEC"}
})

In nginx.conf:

        location /redis {
            internal;
            redis2_raw_queries $args $echo_request_body;
            redis2_pass redis_server;
        }

[agentzh]

Okay, now I get it. This is because the ngx_redis2 module cannot parse recursive redis multi-bulk replies and there is no quick way to add that.

You're recommended to switch to the new lua-resty-redis library that is based on ngx_lua's cosocket API:

https://github.com/agentzh/lua-resty-redis

This library supports the recursive redis multi-bulk replies and is usually more performant than the old ngx_redis2 + lua-resty-parser + ngx.location.capture approach.

Below is a tested example using lua-resty-redis that is directly translated from your example above:

location /t {
    content_by_lua '
        local cjson = require "cjson"
        local redis = require "resty.redis"
        local red = redis:new()

        red:set_timeout(1000) -- 1 sec

        local ok, err = red:connect("127.0.0.1", $TEST_NGINX_REDIS_PORT)
        if not ok then
            ngx.say("failed to connect: ", err)
            return
        end

        local redis_key = "foo"

        local ok, err = red:multi()
        if not ok then
            ngx.say("failed to run multi: ", err)
            return
        end
        ngx.say("multi ans: ", cjson.encode(ok))

        local ans, err = red:sort("log", "by", redis_key .. ":*->timestamp")
        if not ans then
            ngx.say("failed to run sort: ", err)
            return
        end
        ngx.say("sort ans: ", cjson.encode(ans))

        ans, err = red:exec()
        ngx.say("exec ans: ", cjson.encode(ans))

        local ok, err = red:set_keepalive(0, 1024)
        if not ok then
            ngx.say("failed to put the current redis connection

into pool: ", err) return end '; }

And then GET /t gives

multi ans: "OK"
sort ans: "QUEUED"
exec ans: [{}]

You can try it out on your side :)

And another example using lua-resty-redis to do redis transactions can be seen from lua-resty-redis's test suite:

https://github.com/agentzh/lua-resty-redis/blob/master/t/transaction.t#L84

Best regards, -agentzh

[daurnimator]

Okay, I changed to using lua-resty-redis. I'll leave the issue open, as this is still a missing feature of this library.

[AjeetK]

Getting same problem. Any updates on this without using lua-resty-redis?

[agentzh]

@AjeetK I have no interest in adding recursive bulk reply parsing myself since there's better alternative. If you think otherwise, you're welcome to submit a pull request for it. Thank you.

[daurnimator]

FWIW I ended up moving to my own library (https://github.com/daurnimator/lredis).

It needs a patch to openresty (https://github.com/openresty/lua-nginx-module/pull/450) to use the inbuilt sockets in a non-blocking manner.

However if you just need redis reply parsing, you can use just that bit: https://github.com/daurnimator/lredis/blob/master/lredis/protocol.lua#L40

[agentzh]

@daurnimator What's wrong with lua-resty-redis?

[daurnimator]

@daurnimator What's wrong with lua-resty-redis?

• doesn't work outside of nginx (I use the same redis-using code from both inside nginx and other applications. not to mention testing)
• ~~doesn't support subscribe mode~~ looks like it does now.
• pipelining doesn't work how I'd like

See also: https://github.com/daurnimator/lredis#why-not-_________

[agentzh]

@daurnimator The 1st one is irrelevant to lua-resty-redis since that library is created specifically for OpenResty. And this GitHub issue is for an OpenResty specific component, ngx_redis2. I don't see how an ngx_redis2 user would care about other Lua execution environments if he cares about Lua at all :)

The 2nd one is invalid, as you have found out already.

The 3rd one is too vague and subjective, which is not helpful at all :)

[daurnimator]

@daurnimator The 1st one is irrelevant to lua-resty-redis since that library is created specifically for OpenResty. And this GitHub issue is for an OpenResty specific component, ngx_redis2.

you asked :P

[agentzh]

@daurnimator Sure. Just wondering about what's wrong with lua-resty-redis :)