ab并发压测下的lock超时

[liangwangkai]

redis连接参数： idletimeout=1 keepaliveTimeout=60000 keepaliveSize=300 connectionTimout=1000 maxRedirection=5

ab压测： ab -c 10 -n 50 无报错

ab -c 10 -n 100 产生下面报错： `2019/08/26 19:16:13 [error] 60698#5338817: *962 [lua] rediscluster.lua:168: init_slots(): failed to acquire the lock in initialization slot cache: timeout, client: 127.0.0.1, server: ***, request: "GET /hehe/init?ok=2222000&version=7.2.0&source=2 HTTP/1.0", host: "localhost:9110"

2019/08/26 19:16:13 [error] 60698#5338817: 963 [lua] rediscluster.lua:168: init_slots(): failed to acquire the lock in initialization slot cache: timeout, client: 127.0.0.1, server:**, request: "GET /hehe/init?ok=2222000&version=7.2.0&source=2 HTTP/1.0", host: "localhost:9110"

2019/08/26 19:16:13 [error] 60698#5338817: 964 [lua] rediscluster.lua:168: init_slots(): failed to acquire the lock in initialization slot cache: timeout, client: 127.0.0.1, server:**, request: "GET /hehe/init?ok=2222000&version=7.2.0&source=2 HTTP/1.0", host: "localhost:9110"`

查找代码，定位到是 line 166 获取lock 超时： local elapsed, err = lock:lock("redis_cluster_slot_" .. sel f.config.name) if not elapsed then ngx.log(ngx.ERR, "failed to acquire the lock in initial ization slot cache: ", err) return end

大家知道什么原因吗？

[wangrzneu]

You can try this. @liangwangkai

-- redis_conn.lua

local redis_cluster = require("rediscluster") local redis_config = require("redis_config") local redis_conn = redis_cluster:new(redis_config) return redis_conn

-- other file local red_c = require("redis_conn")

[liangwangkai]

You can try this. @liangwangkai

-- redis_conn.lua

local redis_cluster = require("rediscluster") local redis_config = require("redis_config") local redis_conn = redis_cluster:new(redis_config) return redis_conn

-- other file local red_c = require("redis_conn")

have a error :

code : ` local redis_cluster = require "rediscluster"

local config = { name = 'rediscluster', --rediscluster name serv_list = { {ip = "127.0.0.1", port = "6379"}}, --redis cluster node list(host and port), keepalive_timeout = 60000, --redis connection pool idle timeout keepalive_cons = 100, --redis connection pool size connection_timout = 1000, --timeout while connecting max_redirection = 5, --maximum retry attempts for redirection }

local RedisClusterCon = redis_cluster:new(config)

return RedisClusterCon`

error: attempt to yield across C-call boundary

[wangrzneu]

You can try this. @liangwangkai -- redis_conn.lua local redis_cluster = require("rediscluster") local redis_config = require("redis_config") local redis_conn = redis_cluster:new(redis_config) return redis_conn -- other file local red_c = require("redis_conn")

have a error :

code : ` local redis_cluster = require "rediscluster"

local config = { name = 'rediscluster', --rediscluster name serv_list = { {ip = "127.0.0.1", port = "6379"}}, --redis cluster node list(host and port), keepalive_timeout = 60000, --redis connection pool idle timeout keepalive_cons = 100, --redis connection pool size connection_timout = 1000, --timeout while connecting max_redirection = 5, --maximum retry attempts for redirection }

local RedisClusterCon = redis_cluster:new(config)

return RedisClusterCon`

error: attempt to yield across C-call boundary

Sorry, it is my problem. The reason can be found in https://github.com/openresty/lua-resty-redis/issues/3 Can you show the nginx.conf ?

[liangwangkai]

You can try this. @liangwangkai -- redis_conn.lua local redis_cluster = require("rediscluster") local redis_config = require("redis_config") local redis_conn = redis_cluster:new(redis_config) return redis_conn -- other file local red_c = require("redis_conn")

have a error : code : local redis_cluster = require "rediscluster" local config = { name = 'rediscluster', --rediscluster name serv_list = { {ip = "127.0.0.1", port = "6379"}}, --redis cluster node list(host and port), keepalive_timeout = 60000, --redis connection pool idle timeout keepalive_cons = 100, --redis connection pool size connection_timout = 1000, --timeout while connecting max_redirection = 5, --maximum retry attempts for redirection } local RedisClusterCon = redis_cluster:new(config) return RedisClusterCon error: attempt to yield across C-call boundary

Sorry, it is my problem. The reason can be found in openresty/lua-resty-redis#3 Can you show the nginx.conf ?

[liangwangkai]

You can try this. @liangwangkai -- redis_conn.lua local redis_cluster = require("rediscluster") local redis_config = require("redis_config") local redis_conn = redis_cluster:new(redis_config) return redis_conn -- other file local red_c = require("redis_conn")

have a error : code : local redis_cluster = require "rediscluster" local config = { name = 'rediscluster', --rediscluster name serv_list = { {ip = "127.0.0.1", port = "6379"}}, --redis cluster node list(host and port), keepalive_timeout = 60000, --redis connection pool idle timeout keepalive_cons = 100, --redis connection pool size connection_timout = 1000, --timeout while connecting max_redirection = 5, --maximum retry attempts for redirection } local RedisClusterCon = redis_cluster:new(config) return RedisClusterCon error: attempt to yield across C-call boundary

Sorry, it is my problem. The reason can be found in openresty/lua-resty-redis#3 Can you show the nginx.conf ?

`-----dev.conf lua_shared_dict idevz 20m; lua_shared_dict redis_cluster_slot_locks 100m;

server { server_name openresty.test.cn 127.0.0.1; lua_code_cache off; root /data/wwwroot/openresty/api; listen 9110; set $APP_NAME 'openresty'; set $VANILLA_VERSION '0_1_0_rc7'; set $VANILLA_ROOT '/data/wwwroot/openresty/vanilla/framework'; set $template_root ''; set $va_cache_status ''; set $VA_DEV on;

location / {
    content_by_lua_file $document_root/pub/index.lua;
}

}

---nginx.conf

worker_processes 1;

events { worker_connections 1024; }

http { include mime.types; default_type text/html; resolver 127.0.0.1;

sendfile           on;
tcp_nopush         on;
tcp_nodelay        on;

keepalive_timeout  60;

gzip               on;
gzip_vary          on;

gzip_comp_level    6;
gzip_buffers       16 8k;

gzip_min_length    1000;
gzip_proxied       any;
gzip_disable       "msie6";

gzip_http_version  1.0;

gzip_types         text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript image/svg+xml;

lua_package_path "/?.lua;/?/init.lua;/data/wwwroot/openresty/vanilla/framework/?.lua;/data/wwwroot/openresty/vanilla/framework/?/init.lua;;";
lua_package_cpath "/?.so;/data/wwwroot/openresty/vanilla/framework/?.so;;";
init_by_lua_file /data/wwwroot/openresty/vanilla/framework/init.lua;
include dev_vhost/*.conf;

} `

[wangrzneu]

You can try lua_code_cache off; -> lua_code_cache on;

[liangwangkai]

You can try lua_code_cache off; -> lua_code_cache on;

oh！the problem is solved after lua_code_cache on. thanks

[wangrzneu]

Because if lua_code_cache is off, the slot_cache will always miss (rediscluster.lua: 154) and fetch slots from redis for each request, the lock may be timeout in high concurrency (rediscluster.lua: 163).

[membphis]

@liangwangkai please use English in public channel