gatekeeper icon indicating copy to clipboard operation
gatekeeper copied to clipboard

Allocate Lua states on NUMA nodes that GT instances reside

Open mengxiang0811 opened this issue 5 years ago • 3 comments

In GT block, each GT instance needs to keep one Lua state to maintain the most recent Lua policies. Currently, the Lua states are allocated by calling luaL_newstate(). However, this function doesn't allow us to create Lua states on a specific NUMA node, so it's possible that a GT instance and its corresponding Lua state are in different NUMA nodes, which may degrade the performance of Grantor servers. To improve the Grantor performance, we need to find ways to allocate Lua states on a specific NUMA node.

mengxiang0811 avatar Nov 15 '18 16:11 mengxiang0811

One can do this using lua_newstate() instead of luaL_newstate(). One find more information about lua_newstate() in the second edition of the book "Programming in Lua", Chapter "31 Memory Management", Section "31.1 The Allocation Function".

AltraMayor avatar Apr 24 '19 18:04 AltraMayor

The patch https://github.com/AltraMayor/gatekeeper/commit/c5959c06f99e83eb614439508e3f69eca08fe0fa almost closes this issue, but it doesn't do so because the current version of DPDK that we are using does not include rte_realloc_socket().

Once we upgrade our DPDK version, one must replace rte_realloc() with rte_realloc_socket() in alloc_lua_mem_in_dpdk() to close this issue.

AltraMayor avatar Apr 26 '19 13:04 AltraMayor

The patch https://github.com/AltraMayor/gatekeeper/commit/c5959c06f99e83eb614439508e3f69eca08fe0fa had to be disabled because LuaJIT does not support lua_newstate() on 64-bit targets. See issue #261 for details. Until LuaJIT finds a solution for this problem and we update Gatekeeper's version of DPDK, this issue can't be solved.

AltraMayor avatar May 20 '19 13:05 AltraMayor