Redis to JSON

[docyes]

Not really an issue but more so a question. Is it possible to convert redis output to JSON similar to the RDS to JSON with the postgres module?

Thanks, Carl

[agentzh]

This has been a TODO in ngx_redis2 (if you're willing to provide a patch, please let me know). For now, you need to use ngx_lua and lua-redis-parser and lua-cjson to do such conversion:

http://wiki.nginx.org/HttpLuaModule
http://wiki.nginx.org/LuaRedisParser
http://www.kyne.com.au/~mark/software/lua-cjson.php

BTW, all of these components are included and enabled by default in our ngx_openresty bundle:

http://openresty.org/

Regards, -agentzh

[kn007]

章哥，这个功能会完成不？ 还是说要用回2楼方法。 祝假期愉快！

[agentzh]

@kn007 Please, no Chinese here. Now it's recommended to use the lua-resty-redis library plus lua-cjson for such things:

https://github.com/openresty/lua-resty-redis http://www.kyne.com.au/~mark/software/lua-cjson.php

Both of these are standard components in the OpenResty bundle.

[kn007]

Sorry about that. This is also what I know. Thank you.

[mikhailov]

@agentzh I'm trying to build Nginx with 2 modules you recommended (lua-resty-redis, lua-cjson). Alongside with Lua Jit and ngx_devel_kit, but it can't compile:

make: *** No rule to make target `build', needed by `default'.  Stop.
make: *** No rule to make target `install'.  Stop.

I'm obviously doing something wrong but try to follow your installation guide from here: https://github.com/openresty/lua-nginx-module#installation

The installation script I use is here: https://github.com/mikhailov/push-stream-demo/blob/master/nginx-version

OpenResty is an option but currently I only need to have Redis with JSON support.

[mikhailov]

@agentzh is lua-cjson still recommended to use? https://github.com/bungle/lua-resty-libcjson/issues/3#issuecomment-69361355 here I can see lua-resty-json is better, is that correct?

[agentzh]

@mikhailov Please use the OpenResty bundle directly which comes with all the libraries that I mentioned. See

https://openresty.org/#Download

Regards, -agentzh

[mikhailov]

@agentzh OK, I will do that. The only question is Homebrew formula have no lua-redis-libcjson, nor lua-redis (--with-lua-redis or something like that). Probably I'm wrong and the formula has everything by default, is that correct?

[agentzh]

@mikhailov You're recommended to build OpenResty from the latest source tarball yourself. You can find instructions for building on Mac OS X here:

https://openresty.org/#Installation

FYI the official OpenResty team does not maintain any homebrew formulae (yet).

[agentzh]

@mikhailov Those libraries and modules are enabled in the official OpenResty build by default.

[mikhailov]

got it, thank you!

[bungle]

@mikhailov I also recommend using bundled Lua cJSON (aka fork of this: http://www.kyne.com.au/~mark/software/lua-cjson.php), see it here: https://github.com/openresty/lua-cjson

It is a good all around JSON encoder and decoder.

My lua-resty-libcjson is a FFI binding to totally different library. Encoding in my lib sucks badly, but decoding is great. My lib has some tweaks, and in general I think it does things "more correct", especially how it handles arrays and objects. But if you don't have any reason to use it, please use the bundled one.

I have also build others, mostly decoders, just for fun. But some of them are actually freaking fast on stream parsing. If you are looking for faster decoder, there is also CloudFlares lua-resty-json: https://github.com/cloudflare/lua-resty-json

But I don't see much use to that either. If you are not parsing or encoding say multi-megabyte JSON documents, the bundled one is just fine.

[lloydzhou]

demo

using cjson and lua-redis-parser in body_filter_by_lua_block

    location = /redis {                                                     
        redis2_raw_query "$request_body\r\n";
        redis2_pass redis:6379;                                                                  
                                                                                     
        body_filter_by_lua_block {
          local chunk, eof = ngx.arg[1], ngx.arg[2]                                         
          local buffered = ngx.ctx.buffered         
          if not buffered then               
            buffered = {}                             
            ngx.ctx.buffered = buffered
          end                     
          if chunk ~= "" then                      
             buffered[#buffered + 1] = chunk        
             ngx.arg[1] = nil                
          end                                         
          if eof then                  
            local reply = table.concat(buffered)
            ngx.ctx.buffered = nil                    
            local parser = require "redis.parser"   
            local cjson = require "cjson"    
            local res, typ = parser.parse_reply(reply)
            ngx.arg[1] = cjson.encode(res)
          end                                   
        }                                          
    }