stream-lua-nginx-module shared dict feature

globally shared dict instance - accesable from both http and stream contexts

Jun 13 '16 21:06 alonbg

@alonbg This is still a TODO and will get addressed in the near future.

Jun 13 '16 21:06 agentzh

Awesome, I was just wondering if something like this exists or was being worked on.I keep my eye out for it then.

Jun 20 '16 20:06 jebabcock

@agentzh since http and stream modules have thier own private L state. I'm curious how are you going to address this. Thanks!

Aug 01 '16 15:08 alonbg

@alonbg Well, shared memory zones are not coupled with Lua VM states at all. So sharing can be achieved by a higher-level shared nginx C module which can be called something like ngx_meta_lua_module :)

Aug 01 '16 19:08 agentzh

Silly question: if its globally shared across the nginx server instance, why does one need explicit support for shared dict in http or stream context? Shouldn't the nginx architecture (wrt data sharing across workers) be independent of the type of contexts (http/tcp/udp)? The same applies to nginx variables as well. otherwise it seems like awful duplication of effort. Please correct me if my assumptions about nginx internal arch are incorrect.

Aug 10 '16 12:08 rshriram

@rshriram Well, the shm zone API is generic, but it has to be exposed and get utilized by different nginx subsystems, for obvious reasons. Regarding code duplication, well, nginx prefers best performance over avoiding code duplication. Still, I'm devising a macro-level mechanism to share as much code as possible between ngx_stream_lua_module and ngx_http_lua_module, at least from the developers' perspective (from the C compiler's perspective code is still kinda duplicate though with important differences). Well, your question actually goes beyond my decisions, since it's more about nginx core's architectural design.

Aug 10 '16 19:08 agentzh

@agentzh Silly idea, if I may: couldn't we extend the current http/stream shm zone implementation to have a global/local toggle ? (e.g. lua_shared_dict from_http 10m g; g = global) And if needed ngx_meta_lua_module would be a passive bridge between the two modules. A module (be it stream or http) will register it's global shm zone in ngx_meta_lua_module for the other module to add to it's own lmcf->shm_zones. Then again this just might be a really wrong idea :)

And anyway, If you think it's worth kick starting a repo with some boiler plate skeleton code i'll happily fork :)

What do you think ?

Aug 24 '16 15:08 alonbg

@alonbg I'm not sure about the details at this point, unfortunately. But you're more than welcome to fiddle about with various different options in your own branches and send us feedback :)

Aug 24 '16 18:08 agentzh

@agentzh here you go :) Please have a look.

Sep 07 '16 17:09 alonbg

👍 waiting for this feature so I don't have to build a standalone TCP server and move cache to Redis :(