Fix : Lua Bufferoverflow

[KIMDONGYEON00]

Summary

• CVE-2024-31449 was found in Redis, and the same behavior is reproduced in Wireshark.
• A Lua-bit stack overflow causes a crash.
• According to the Redis security advisory, this vulnerability can lead to RCE attacks
• The same vulnerability was also present in Wireshark, and I reported it and it was fixed. https://gitlab.com/wireshark/wireshark/-/issues/20475

What is the current bug behavior?

Located in epan/wslua/lua_bitop.c

static int bit_tohex(lua_State *L)
{
  UBits b = barg(L, 1);
  SBits n = lua_isnone(L, 2) ? 8 : (SBits)barg(L, 2);
  const char *hexdigits = "0123456789abcdef";
  char buf[8];
  int i;
  if (n < 0) { n = -n; hexdigits = "0123456789ABCDEF"; }
  if (n > 8) n = 8;
  for (i = (int)n; --i >= 0; ) { buf[i] = hexdigits[b & 15]; b >>= 4; }
  lua_pushlstring(L, buf, (size_t)n);
  return 1;
}

• The function static int bit_tohex(lua_State *L) sets b from the first argument and n from second (defaulting to 8 if missing).

if (n < 0) { n = -n; hexdigits = "0123456789ABCDEF"; }

• At #L108, if n is negative, the code tries to make it positive.
• But an int only stores values from -2,147,483,648 to 2,147,483,647.
• When you input -2,147,483,648 and compute n = -n , the result should be 2,147,483,648.
• However this number is too big for an int .
• So an intger overflow occurs and n stays as -2,147,483,648.

for (i = (int)n; --i >= 0; ) { buf[i] = hexdigits[b & 15]; b >>= 4; } //buf[0xffffffff]

• This flaw bypasses the (n > 8) check at #L140.
• At #L110, a negative n makes the loop access buf[0xffffffff] and crash.
• This bug was found in Redis and is labeled CVE-2024-31449.
• Add the following code as in the [redis commit.](https://github.com/redis/redis/commit/3a2669e8aea6723a7a3fb9daf0ea888435df9191)

if (n == INT32_MIN) n = INT32_MIN+1;

• This code is a condition that prevents integer overflow.
• Adding a check to stop integer overflow prevents the stack buffer from overflowing.

What is the expected correct behavior?

if (n == INT32_MIN) n = INT32_MIN+1;

• Inserting this code above line #L108 prevents an integer overflow.

"0000FFFF"

• Then, it returns a correct value without crashing.