module do not support with PCRE2 on nginx 1.21.5

[kn007]

Compile failed.

Nginx 1.21.5: now nginx is built with the PCRE2 library by default.

/tmp/nginx.lJpu0R.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_malloc'
/tmp/nginx.lJpu0R.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.lJpu0R.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_malloc'
/tmp/nginx.lJpu0R.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.lJpu0R.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_body_filter_by_chunk: error: undefined reference to 'pcre_malloc'
/tmp/nginx.lJpu0R.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_body_filter_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.lJpu0R.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_body_filter_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.lJpu0R.ltrans12.ltrans.o:<artificial>:function ngx_http_lua_pcre_malloc_done: error: undefined reference to 'pcre_malloc'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_destroy_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_destroy_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_exec_regex: error: undefined reference to 'pcre_dfa_exec'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_exec_regex: error: undefined reference to 'pcre_exec'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_free'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_alloc'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_alloc'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_free'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_compile'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_assign_jit_stack'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.lJpu0R.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_pcre_version: error: undefined reference to 'pcre_version'

==============

it's time https://github.com/openresty/lua-nginx-module/issues/577#issuecomment-146131885

[yangzhaofeng]

Here is a workaround:

add -lpcre in --with-ld-opt when running configure, for example

./configure --with-compat --with-ld-opt='-lpcre' --add-dynamic-module=../ngx_devel_kit-${DEVEL_KIT_VERSION} --add-dynamic-module=../lua-nginx-module-${LUA_MODULE_VERSION}

[kn007]

@yangzhaofeng Yes, I I have do this 5 days ago. But this is not the solution I want.

[steamboatid]

i can confirm this error w/ nginx 1.21.6

[pgnd]

@yangzhaofeng

Can you comment as to if lua-nginx-module pcre2 support is intended/planned? and for which milestone?

Given @agentzh 's 2015 comment:

"API will not migrate to PCRE2 unless the nginx core does that as well"

your offered 'workaround', and that the pcre -> pcre2 API migration is likely non-trivial, it's really not clear what the current plan is.

would be helpful to know particularly for those choosing to use current/next nginx versions.

[pgnd]

for anyone interested,

an alternative that covers my scripting needs is migrating current lua+nginx functionality over to njs.

it's closely aligned to upstream nginx, clearly, building nicely with latest nginx, pcre2, etc.

and for my needs looks like i'll be able to cover all existing lua-script functionality.

not for everyone, but trimming down to one scripting environment is attractive ...

[eilandert]

any idea if there are plans to fix this?

[cleverpig]

any idea if there are plans to fix this? keep watching.

[zhuizhuhaomeng]

will support pcre2 when upgrading to version 1.21.6

[kn007]

will support pcre2 when upgrading to version 1.21.6

@zhuizhuhaomeng

Compile failed with nginx 1.21.6

/tmp/nginx.3QZ5G3.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_malloc'
/tmp/nginx.3QZ5G3.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.3QZ5G3.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_malloc'
/tmp/nginx.3QZ5G3.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_log_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.3QZ5G3.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_body_filter_by_chunk: error: undefined reference to 'pcre_malloc'
/tmp/nginx.3QZ5G3.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_body_filter_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.3QZ5G3.ltrans10.ltrans.o:<artificial>:function ngx_http_lua_body_filter_by_chunk: error: undefined reference to 'pcre_free'
/tmp/nginx.3QZ5G3.ltrans12.ltrans.o:<artificial>:function ngx_http_lua_pcre_malloc_done: error: undefined reference to 'pcre_malloc'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_destroy_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_destroy_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_exec_regex: error: undefined reference to 'pcre_dfa_exec'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_exec_regex: error: undefined reference to 'pcre_exec'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_free'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_alloc'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_alloc'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_set_jit_stack_size: error: undefined reference to 'pcre_jit_stack_free'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_compile'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_assign_jit_stack'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_free_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_study'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_compile_regex: error: undefined reference to 'pcre_fullinfo'
/tmp/nginx.3QZ5G3.ltrans13.ltrans.o:<artificial>:function ngx_http_lua_ffi_pcre_version: error: undefined reference to 'pcre_version'

[kn007]

How is the progress going?

Any plan to resolve?

[xiaocang]

Hi @kn007, we will probably support pcre2 when openresty is upgraded to 1.21.6.x (based on nginx 1.21.6). It looks like some of these modules need to have some patches applied for them to work properly

[msva]

@xiaocang can we have any estimated timeframe for when it will happen, please?

[xiaocang]

@msva Perhaps in the next quarter, and relevant PR would be welcome.

[bagasik]

Hello. nginx-1.22 lua-resty-core-0.1.23 not working.

My system FreeBSD 12.3-RELEASE-p5 amd64. Installed ports: nginx-1.22.0,2 lua-resty-core-0.1.23 pcre-8.45_1 pcre2-10.40

Error: nginx -t nginx: [emerg] dlopen() "/usr/local/libexec/nginx/ngx_http_lua_module.so" failed (/usr/local/libexec/nginx/ngx_http_lua_module.so: Undefined symbol "pcre_free") in /usr/local/etc/nginx/nginx.conf:2 nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed

[jirutka]

This happens when you build nginx with pcre2, but the Lua module with pcre. These cannot be mixed. Sadly, you have to build nginx withou pcre2 to use the Lua module.

[bagasik]

How can I make nginx without pcre2?

[bagasik]

nginx is built --with-pcre.

nginx -V nginx version: nginx/1.20.2 built with OpenSSL 1.1.1l-freebsd 24 Aug 2021 TLS SNI support enabled configure arguments: --prefix=/usr/local/etc/nginx --with-cc-opt='-I /usr/local/include' --with-ld-opt='-L /usr/local/lib' --conf-path=/usr/local/etc/nginx/nginx.conf --sbin-path=/usr/local/sbin/nginx --pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx/error.log --user=www --group=www --with-compat --modules-path=/usr/local/libexec/nginx --with-file-aio --http-client-body-temp-path=/var/tmp/nginx/client_body_temp --http-fastcgi-temp-path=/var/tmp/nginx/fastcgi_temp --http-proxy-temp-path=/var/tmp/nginx/proxy_temp --http-scgi-temp-path=/var/tmp/nginx/scgi_temp --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi_temp --http-log-path=/var/log/nginx/access.log --with-http_v2_module --with-http_addition_module --with-http_auth_request_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_realip_module --with-pcre --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-cc-opt='-DNGX_HAVE_INET6=0 -I /usr/local/include' --without-mail_imap_module --without-mail_pop3_module --without-mail_smtp_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-stream=dynamic --add-dynamic-module=/mnt/111/nginx/work/ngx_devel_kit-0.3.1 --add-dynamic-module=/mnt/111/nginx/work/lua-nginx-module-0.10.20

[jirutka]

How can I make nginx without pcre2?

--without-pcre2

[dvershinin]

This happens when you build nginx with pcre2, but the Lua module with pcre. These cannot be mixed. Sadly, you have to build nginx withou pcre2 to use the Lua module.

You can mix them in this way, as long as -lpcre is added (as mentioned in one of the previous comments).

[bagasik]

Hello. When will the situation in (nginx+lua) with pcre2 be fixed?

Didn't understand about -lpcre. Please explain in more detail. What is -lpcre ?

[dvershinin]

@bagasik this comment explains it clearly. What problems are you having?

[bagasik]

What is this -lpcre parameter. Do I need to rebuild the pcre2 port with this parameter?

[earas]

I am having the same problem. nginx 1.21.5 and 1.21.6 comes with PCRE2. And Lua is incompatible with that. For QUIC, I need PCRE2 support and version 1.21.5 or higher. So I can't downgrade nginx. So that I have to wait for Lua module with PCRE2. Will you fix this issue in the next update? Do you have any plan for PCRE2?

[splitice]

@earas lua-nginx-module doesnt support http3 anyway. https://github.com/openresty/openresty/issues/741

[xiaocang]

I am having the same problem. nginx 1.21.5 and 1.21.6 comes with PCRE2. And Lua is incompatible with that.

@earas, You're right, to support PCRE2 some more PRs are needed

[borats]

You're right, to support PCRE2 some more PRs are needed

@xiaocang perhaps have a milestone where people fluent in Go/Lua, *NIX, Build infra can contribute to the effort of aligning upstream version more easily with kubernetes/ingress-nginx. I believe, the critical mass of kubernetes/ingress-nginx is now at a point where we need to move this needle more rapidly than authors envisioned. Personally, sponsorship of project/s could potentially help too.

[jirutka]

I’d suggest moving from nginx Lua to njs (NGINX JavaScript). It’s being developed directly by NGINX, so better supported. I’ve created tooling for developing and testing njs extensions in TypeScript (or plain JavaScript). You can start with njs-typescript-starter (note: it’s a bit outdated now, njs already supports async/await, but it should work; I’ll update it later).

[kn007]

@jirutka The njs does not support I/O operations, execute external programs...

[splitice]

njs has very different design parameters to openresty (and Lua support in general).

I'll be the last person to state that the openresty ecosystem community (developer contributions, merging and triage) is well run, well resourced and working optimally but irrespective of that it's a good platform. The base is good it's the community aspects that have failed.

I could supply example pr's from the past. There are many examples of complete features with real applications that never merged (incr extentions and cas come to mind).

[leeiitb]

Waiting for PCRE2 support 🤔 Thanks