Core dumps with Lua nginx module and OCSP enabled on ingress-nginx

[strongjz]

Ingress-nginx updated the lua-nginx-module to version b5d1688ae722538ba4dc8a7ec08820a08abfb93d and started seeing core dumps only when OCSP is enabled.

nginx version: 1.25.5 lua jit: v2.1-20240314 lua Resty Core: v0.1.28 Alpine: 3.20.0

All versions of modules running: https://github.com/kubernetes/ingress-nginx/blob/release-1.11/images/nginx-1.25/rootfs/build.sh

We reverted to it to known good version v0.10.26 and the core dump issues with OCSP went away.

To reproduce, run ingress-nginx version 1.11.0 with enable-ocsp: true https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#enable-ocsp and test it with a cert manager and an HTTPS enabled ingress.

Core Dumps

sudo gdb ./nginx core/core.99

...
#0  0x00005626f775dd3e in ngx_http_lua_ffi_ssl_validate_ocsp_response (resp=<optimized out>, resp_len=<optimized out>, chain_data=<optimized out>, chain_len=<optimized out>, errbuf=0x7fa407769500 "schemeupstream_name\005", errbuf_size=0x7fa4077ad818, valid=0x600)
    at /tmp/build/lua-nginx-module/src/ngx_http_lua_ssl_ocsp.c:483

...
(gdb) backtrace
#18 0x00005626f775db20 in ?? () at /tmp/build/lua-nginx-module/src/ngx_http_lua_ssl_ocsp.c:235

lldb -c core.766
(lldb) target create --core "core.766"
Core file '/Users/strongjz/go/src/github/kubernetes/ingress-nginx/core.766' (x86_64) was loaded.
(lldb) bt all
* thread #1, name = 'nginx', stop reason = signal SIGSEGV: address not mapped to object
  * frame #0: 0x00005626f775dd3e nginx`ngx_http_lua_ffi_ssl_validate_ocsp_response(resp=<unavailable>, resp_len=<unavailable>, chain_data=<unavailable>, chain_len=<unavailable>, errbuf="0S0Q0O0M0K0\t\U00000006\U00000005+\U0000000e\U00000003\U00000002\U0000001a\U00000005", errbuf_size=0x00007fa4077ad818, valid=0x0000000000000600) at ngx_http_lua_ssl_ocsp.c:388:16
    frame #1: 0x00007fa40eb68f92

Please see https://github.com/kubernetes/ingress-nginx/issues/11588 for more information.

Core dumps are available at

• https://drive.google.com/file/d/1wKiIQe4RiiXZT95hgE5nn3TcQYhUYwfT/view?usp=sharing
• https://drive.google.com/file/d/1wKiIQe4RiiXZT95hgE5nn3TcQYhUYwfT/view?usp=sharing

[zhuizhuhaomeng]

@strongjz I don't have the ENV to test ocsp.

Would you please test the following patch?

diff --git a/src/ngx_http_lua_ssl_ocsp.c b/src/ngx_http_lua_ssl_ocsp.c
index c9f24cd2..73e1b9c7 100644
--- a/src/ngx_http_lua_ssl_ocsp.c
+++ b/src/ngx_http_lua_ssl_ocsp.c
@@ -280,7 +280,7 @@ ngx_http_lua_ffi_ssl_validate_ocsp_response(const u_char *resp,
     OCSP_RESPONSE         *ocsp = NULL;
     OCSP_BASICRESP        *basic = NULL;
     STACK_OF(X509)        *chain = NULL;
-    ASN1_GENERALIZEDTIME  *thisupdate, *nextupdate;
+    ASN1_GENERALIZEDTIME  *thisupdate = NULL, *nextupdate = NULL;
 
     ocsp = d2i_OCSP_RESPONSE(NULL, &resp, resp_len);
     if (ocsp == NULL) {

Would you please start the nginx with valgrind if the above patch does not work?

[zhuizhuhaomeng]

@strongjz any progress on this issue?

[strongjz]

I apologize for not getting back to you on this, we didnt have time to test it and left the older version enabled. We are looking into the next release to have the version that supports nginx 1.27.