node-mapnik
node-mapnik copied to clipboard
mapnik
call to mapnik.register_default_input_plugins never returns
Call mapnik.register_default_input_plugins() never returns.
- mapnik module version: 4.2.0 (same problem with 4.0.2)
- node version: 10.14.2
- OS: alpine linux, version edge, running in docker
Last lines from strace output:
mprotect(0x25f844204000, 503808, PROT_READ|PROT_WRITE) = 0
mprotect(0x25f844204000, 503808, PROT_READ|PROT_EXEC) = 0
stat("/freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
stat("/freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open("/freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input", O_RDONLY|O_CLOEXEC|O_DIRECTORY) = 20
fcntl(20, F_SETFD, FD_CLOEXEC) = 0
getdents64(20, /* 13 entries */, 2048) = 432
getdents64(20, /* 0 entries */, 2048) = 0
close(20) = 0
stat("/freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input/csv.input", {st_mode=S_IFREG|0755, st_size=1362981, ...}) = 0
stat("/freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input/csv.input", {st_mode=S_IFREG|0755, st_size=1362981, ...}) = 0
futex(0x7f8f8821b0cc, FUTEX_WAIT_PRIVATE, 2147483664, NULL
Removing /freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input/csv.input makes it halt on another *.input file. If I remove all files from /freemap-mapnik/node_modules/mapnik/lib/binding/lib/mapnik/input/ then the call returns but obviously it fails later as necessary drivers are missing.
I've checked all threads of node process. All are waiting on futex/FUTEX_WAIT_PRIVATE but one was epoll_pwait(13, .
Some output from gdb:
(gdb) bt
#0 0x00007f8bfa29cfd1 in ?? () from /lib/ld-musl-x86_64.so.1
#1 0x00007fff49699508 in ?? ()
#2 0x00007f8bf6cd70cc in mapnik::CreateStatic<mapnik::datasource_cache>::create()::static_memory () from /freemap-mapnik/node_modules/mapnik/lib/binding/lib/libmapnik.so
#3 0xffffffff80000010 in ?? ()
#4 0x00007f8bfa29c561 in __timedwait_cp () from /lib/ld-musl-x86_64.so.1
#5 0x0000000000000000 in ?? ()
datasource_cache is the only place in Mapnik where std::recursive_mutex is used: src/datasource_cache.cpp#L163. I'm wondering whether this cannot be somehow connected to your problem. Maybe the mutex actually created is not recursive one.
but one was
epoll_pwait(13,
You can take a look what is behind the file descriptor 13 by ls -l /proc/6482/fd with pid of your app.
Are you building Mapnik directly in Docker? In that case it would be great if you can share the dockerfile.
Dockerfile:
RUN apk add --update nodejs npm gcompat
RUN mkdir /test
WORKDIR /test
RUN npm init -y
RUN npm i --save mapnik
RUN echo "require('mapnik').register_default_input_plugins();" > index.js
Steps:
-
docker build -t test . -
docker run -it -w /test test node .
docker run should terminate but it doesn't.
I still don't know what exactly causes that deadlock, but Mapnik package from Alpine works - no surprise - well:
My test app:
#include <mapnik/datasource_cache.hpp>
#include <iostream>
#undef NDEBUG
#include <cassert>
int main()
{
auto & cache = mapnik::datasource_cache::instance();
assert(cache.register_datasources("/usr/lib/mapnik/input/", true));
for (auto const & name : cache.plugin_names())
{
std::clog << name << std::endl;
}
return 0;
}
Output:
$ ./test
csv
gdal
geojson
ogr
pgraster
postgis
raster
shape
sqlite
topojson
I would maybe just not use the prebuilt Mapnik from npm, rather build it from latest sources inside Docker. It is also way how to deploy patches quickly. To build Mapnik itself is simple, I would like to try also node-mapnik:
FROM alpine:latest
RUN echo "http://repository.fit.cvut.cz/mirrors/alpine/edge/main" > /etc/apk/repositories; \
echo "http://repository.fit.cvut.cz/mirrors/alpine/edge/community" >> /etc/apk/repositories; \
echo "http://repository.fit.cvut.cz/mirrors/alpine/edge/testing" >> /etc/apk/repositories
RUN apk update && apk upgrade
RUN apk add bash make git python gdal-dev cairo-dev \
postgresql-dev harfbuzz-dev g++ boost-dev sqlite-dev \
proj4-dev jpeg-dev tiff-dev libwebp-dev
RUN mkdir /build
WORKDIR /build
RUN git clone https://github.com/mapnik/mapnik.git
WORKDIR mapnik
RUN git checkout v3.0.x
RUN git submodule update --init
RUN ./configure
RUN JOBS=2 make
RUN make install
Building node-mapnik:
FROM mapy-sk-mapnik-build:latest
RUN apk add nodejs npm
WORKDIR /build
RUN git clone https://github.com/mapnik/node-mapnik.git
WORKDIR node-mapnik
RUN git checkout v3.0.x
RUN git submodule update --init
RUN make release_base
Test image:
FROM mapy-sk-mapnik-build-node:latest
RUN mkdir /test
WORKDIR /test
COPY package.json .
RUN npm i
RUN echo "var mapnik = require('mapnik'); mapnik.register_default_input_plugins();console.log(mapnik.datasources());" > index.js
package.json:
{
"name": "test",
"version": "1.0.0",
"description": "",
"dependencies": {
"mapnik": "file:/build/node-mapnik"
},
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC"
}
Output:
$ docker run --rm -i -t mapy-sk-mapnik-build-node-test:latest node .
[ 'csv',
'gdal',
'geojson',
'ogr',
'pgraster',
'postgis',
'raster',
'shape',
'sqlite',
'topojson' ]
Great, thanks!
Anyway we already use debian-based containers but we'll definitively use your instructions if we decide to switch to alpine :-)
Not sure whether I should close this issue now. I'll leave it up on you.