Can't build third-party form input module

[c-mart]

Trying to follow the new instructions to add modules to the Nginx Docker image. The module I need is form input. It depends on the Nginx Development Kit module.

I set this up in the modules/ directory:

# tree ngx_form_input/
ngx_form_input/
├── build-deps
└── source
# cat ngx_form_input/source 
https://github.com/calio/form-input-nginx-module/archive/v0.12.tar.gz
# cat ngx_form_input/build-deps 
libnginx-mod-http-ndk

Then I run:

docker build --build-arg ENABLED_MODULES="ndk ngx_form_input" -t nginx-with-form-input .

It appears to successfully build the development kit (ndk? ngx_devel_kit?), then tries to build the form input module and fails thusly:

adding module in extra/ngxforminput-1.19.6
error: ngx_devel_kit is required to build ngx_form_input; please put it before ngx_form_input.
make: *** [debian/rules:48: config.status.nginx] Error 1
dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui failed
real 10.21
user 7.61
sys 2.10
make: *** [Makefile:149: module-ngxforminput] Error 29
build_module.sh: ERROR: Build failed
The command '/bin/sh -c set -ex     && apt update     && apt install -y --no-install-suggests --no-install-recommends                 patch make wget mercurial devscripts debhelper dpkg-dev                 quilt lsb-release build-essential libxml2-utils xsltproc                 equivs git g++     && hg clone https://hg.nginx.org/pkg-oss/     && cd pkg-oss     && mkdir /tmp/packages     && for module in $ENABLED_MODULES; do         echo "Building $module for nginx-$NGINX_VERSION";         if [ -d /modules/$module ]; then             echo "Building $module from user-supplied sources";             if [ ! -s /modules/$module/source ]; then                 echo "No source file for $module in modules/$module/source, exiting";                 exit 1;             fi;             if [ -f /modules/$module/build-deps ]; then                 echo "Installing $module build dependencies";                 apt update && apt install -y --no-install-suggests --no-install-recommends $(cat /modules/$module/build-deps | xargs);             fi;             if [ -x /modules/$module/prebuild ]; then                 echo "Running prebuild script for $module";                 /modules/$module/prebuild;             fi;             /pkg-oss/build_module.sh -v $NGINX_VERSION -f -y -o /tmp/packages -n $module $(cat /modules/$module/source);         elif make -C /pkg-oss/debian list | grep -P "^$module\s+\d" > /dev/null; then             echo "Building $module from pkg-oss sources";             cd /pkg-oss/debian;             make rules-module-$module BASE_VERSION=$NGINX_VERSION;             mk-build-deps --install --tool="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes" debuild-module-$module/nginx-$NGINX_VERSION/debian/control;             make module-$module BASE_VERSION=$NGINX_VERSION;             find ../../ -maxdepth 1 -mindepth 1 -type f -name "*.deb" -exec mv -v {} /tmp/packages/ \;;         else             echo "Don't know how to build $module module, exiting";             exit 1;         fi;     done' returned a non-zero code: 1

Unsure if this problem lies with the module I'm trying to build, or if I'm naming things incorrectly, or what. The development kit seems to be named "ndk" on https://hg.nginx.org/pkg-oss, but the Debian package (as one specifies in the build-deps file) is named "libnginx-mod-http-ndk". Would appreciate if someone can help me understand!

[thresheek]

Hi @c-mart!

It looks like this module requires ndk module sources to be present (as opposed to binary module as installed by build-deps), and it needs to be built at the same time as form-input.

We dont support that kind of dependency between modules at this moment.

In theory, we can try and guess that a given module depends on ndk in build_module.sh, download & unpack ndk sources and use them while configuring. I suppose the best approach is not to package the ndk .so in the resulting package, but rather add a dependency.

This, however, does not fix the general issue that there can be other build-dependencies between modules, and ndk is just one of them (although widely used).

[thresheek]

FYI I've submitted a relevant fix for build_module.sh for a review, so when/if it lands into pkg-oss, I'll let you know to try and build the module again.

[c-mart]

Thank you @thresheek! In the meantime I'll try to find a workaround to deal without this module.

[thresheek]

So we decided not to include the patch I made for build_module.sh since it only fixed one particular problem with the ndk module, and not for any other dependency that might be there for other modules.

What we think as a proper solution in general is to, sadly, write a Makefile specific for the module you're building and place it alongside pkg-oss other modules. This is undoubtedly quite a bit more work than just launching a script, but allows to solve that issue properly. There is however no support in this repository for that. Maybe I should replace the whole modules/{source,build-deps,prebuild} thingie with just copying a Makefile as specified by the user. Thoughts?

[had1z]

Hi, I have the same problem in building set_misc module. I'm using the Dockerfile for mainline-alpine which you specified in this repository.

adding module in ../setmisc-1.19.10
error: ngx_devel_kit is required to build ngx_set_misc; please put it before ngx_set_misc.
>>> ERROR: nginx-module-setmisc: build failed
Command exited with non-zero status 1
real 12.45
user 7.98
sys 3.77
make: *** [Makefile:179: module-setmisc] Error 1
build_module.sh: ERROR: Build failed

[had1z]

There is a small difference in my case which is I added devel_kit module just like set_misc but before it, instead of using it's package in build-deps file. This is how my modules directory looks like:

├── devel_kit
│   └── source
├── redis
│   └── source
└── set_misc
    └── source

And these are my source files contents: https://github.com/vision5/ngx_devel_kit/archive/refs/tags/v0.3.1.tar.gz https://people.freebsd.org/~osa/ngx_http_redis-0.3.9.tar.gz https://github.com/openresty/set-misc-nginx-module/archive/refs/tags/v0.32.tar.gz

[thresheek]

Hi @had1z, in your case I guess you can fallback to set-misc from pkg-oss (so don't specify it nor devel kit in the filesystem layout), and build only redis from provided sources.

[MuthuChess]

Hi @thresheek ,

I have a similar case where I want set-misc 0.33 but pkg-oss has only v0.32.. set-misc is dependent on ndk.. Both scenarios didn't work

Case 1:

└── set-misc-0.33 └── source

docker-compose.yml ENABLED_MODULES: set-misc-0.33

Case 2:

└── set-misc-0.33 └── source

docker-compose.yml ENABLED_MODULES: ndk set-misc-0.33

adding module in extra/setmisc033-1.21.3 error: ngx_devel_kit is required to build ngx_set_misc; please put it before ngx_set_misc. make: *** [debian/rules:57: config.status.nginx] Error 1 dpkg-buildpackage: error: debian/rules build subprocess returned exit status 2 debuild: fatal error at line 1182: dpkg-buildpackage -us -uc -ui failed real 13.96 user 8.10 sys 2.21 make: *** [Makefile:210: module-setmisc033] Error 29 build_module.sh: ERROR: Build failed ... .... Reading package lists... E: Unsupported file /tmp/packages/nginx-module-setmisc033_1.21.3*.deb given on commandline Service 'web' failed to build: The command '/bin/sh -c set -ex && apt update && . /tmp/packages/modules.env && for module in $BUILT_MODULES; do apt install --no-install-suggests --no-install-recommends -y /tmp/packages/nginx-module-${module}_${NGINX_VERSION}*.deb; done && rm -rf /tmp/packages && rm -rf /var/lib/apt/lists/' returned a non-zero code: 100

Is there a way I can install set-misc 0.33 either with existing ndk in pkg-oss or by manually adding ndk to the modules? Either way is okay but I get the error.

[xiaosongxiaosong]

look like the same error with @MuthuChess

All packages are up to date.
+ . /tmp/packages/modules.env
+ BUILT_MODULES= subs-filter
+ apt install --no-install-suggests --no-install-recommends -y /tmp/packages/nginx-module-subs-filter_1.23.2*.deb

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
E: Unsupported file /tmp/packages/nginx-module-subs-filter_1.23.2*.deb given on commandline
The command '/bin/sh -c set -ex     && apt update     && . /tmp/packages/modules.env     && for module in $BUILT_MODULES; do            apt install --no-install-suggests --no-install-recommends -y /tmp/packages/nginx-module-${module}_${NGINX_VERSION}*.deb;        done     && rm -rf /tmp/packages     && rm -rf /var/lib/apt/lists/'returned a non-zero code: 100

sudo docker build --build-arg ENABLED_MODULES="subs-filter" -t nginx-with-subs-filter . > ../log.txt  2>&1 &

[xiaosongxiaosong]

My problem is caused by network problems, using GitHub actions can be successful https://github.com/xiaosongxiaosong/nginx-with-subs-filter

look like the same error with @MuthuChess

All packages are up to date.
+ . /tmp/packages/modules.env
+ BUILT_MODULES= subs-filter
+ apt install --no-install-suggests --no-install-recommends -y /tmp/packages/nginx-module-subs-filter_1.23.2*.deb

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
E: Unsupported file /tmp/packages/nginx-module-subs-filter_1.23.2*.deb given on commandline
The command '/bin/sh -c set -ex     && apt update     && . /tmp/packages/modules.env     && for module in $BUILT_MODULES; do            apt install --no-install-suggests --no-install-recommends -y /tmp/packages/nginx-module-${module}_${NGINX_VERSION}*.deb;        done     && rm -rf /tmp/packages     && rm -rf /var/lib/apt/lists/'returned a non-zero code: 100

sudo docker build --build-arg ENABLED_MODULES="subs-filter" -t nginx-with-subs-filter . > ../log.txt  2>&1 &