consbio
Memory leak
@olehz thanks for reporting and for investigating the potential underlying issue from mapbox-gl-native.
Can you tell us more about your usage here to help us investigate?
- what version of
mbgl-renderer? - what OS / container environment?
- what is the load you are throwing at
mbgl-renderer- is it sporadic image render requests, are you generating many images in a pipeline (e.g., rendering tiles), etc?
Am I seeing correctly that you have several node processes of mbgl-renderer listening on the same port?
Are you able to reproduce this on NodeJS 10?
There are many requests for generating images in the pipeline. The data source is quite large GeoJSON. I am not able to test this on NodeJS 10.
I also found a similar problem. As the number of renders increases and the style is a larger geojson, the memory usage will gradually increase without being released. I don’t know if it’s an upstream problem.
use docker for consbio/mbgl-renderer
Looks like there are indeed memory leaks coming from mapbox-gl-native. Here's a bit of valgrind profiling of the NodeJS 10 version:
==315== 5,310 bytes in 253 blocks are definitely lost in loss record 5,914 of 5,936
==315== at 0x483877F: malloc (vg_replace_malloc.c:307)
==315== by 0x4C3AE4A: strdup (strdup.c:42)
==315== by 0x92EB4A6: ???
==315== by 0x92EB254: ???
==315== by 0x92907CC: ???
==315== by 0x9209436: ??? (in /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0.0.0)
==315== by 0x9209CE8: __glDispatchMakeCurrent (in /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0.0.0)
==315== by 0x91877F6: ??? (in /usr/lib/x86_64-linux-gnu/libEGL.so.1.1.0)
==315== by 0xB56DC9D: mbgl::gl::EGLBackendImpl::activateContext() (headless_backend_egl.cpp:124)
==315== by 0xB421CE4: activate (backend_scope.cpp:56)
==315== by 0xB421CE4: mbgl::gfx::BackendScope::BackendScope(mbgl::gfx::RendererBackend&, mbgl::gfx::BackendScope::ScopeType) (backend_scope.cpp:31)
==315== by 0xB3CE8CC: operator() (headless_frontend.cpp:29)
==315== by 0xB3CE8CC: std::_Function_handler<void (), mbgl::HeadlessFrontend::HeadlessFrontend(mbgl::Size, float, mbgl::gfx::ContextMode, std::experimental::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >)::$_0>::_M_invoke(std::_Any_data const&) (functional:1871)
==315== by 0xA711E0: uv__async_io (async.c:147)
==315==
==315== 12,448 (368 direct, 12,080 indirect) bytes in 1 blocks are definitely lost in loss record 5,925 of 5,936
==315== at 0x483877F: malloc (vg_replace_malloc.c:307)
==315== by 0x15E67C9: icu_64::DecimalFormat::clone() const (in /usr/local/bin/node)
==315== by 0x15DEC76: icu_64::NumberFormat::createInstance(icu_64::Locale const&, UErrorCode&) (in /usr/local/bin/node)
==315== by 0x104FBDC: v8::internal::(anonymous namespace)::CreateICUNumberFormat(v8::internal::Isolate*, icu_64::Locale const&, v8::internal::Handle<v8::internal::JSObject>) (in /usr/local/bin/node)
==315== by 0x1051094: v8::internal::NumberFormat::InitializeNumberFormat(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>, v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::JSObject>) (in /usr/local/bin/node)
==315== by 0x11796D6: v8::internal::Runtime_CreateNumberFormat(int, v8::internal::Object**, v8::internal::Isolate*) (in /usr/local/bin/node)
==315== by 0x120F6065C1D7: ???
==315== by 0x120F6068FA14: ???
==315== by 0x120F606118D4: ???
==315== by 0x120F606118D4: ???
==315== by 0x120F606118D4: ???
==315== by 0x120F6060A5C2: ???
==315==
==315== 40,032 (32 direct, 40,000 indirect) bytes in 1 blocks are definitely lost in loss record 5,933 of 5,936
==315== at 0x4838DEF: operator new(unsigned long) (vg_replace_malloc.c:342)
==315== by 0x91C254: node::Buffer::New(v8::Isolate*, char*, unsigned long, void (*)(char*, void*), void*) (in /usr/local/bin/node)
==315== by 0xB2A9D36: NewBuffer (nan.h:890)
==315== by 0xB2A9D36: node_mbgl::NodeMap::renderFinished() (node_map.cpp:542)
==315== by 0xA711E0: uv__async_io (async.c:147)
==315== by 0xA83B07: uv__io_poll (linux-core.c:431)
==315== by 0xA71B6A: uv_run (core.c:375)
==315== by 0x905664: node::Start(v8::Isolate*, node::IsolateData*, std::vector<std::string, std::allocator<std::string> > const&, std::vector<std::string, std::allocator<std::string> > const&) (in /usr/local/bin/node)
==315== by 0x90374E: node::Start(int, char**) (in /usr/local/bin/node)
==315== by 0x4BD3D09: (below main) (libc-start.c:308)
==315==
==315== LEAK SUMMARY:
==315== definitely lost: 9,230 bytes in 329 blocks
==315== indirectly lost: 53,281 bytes in 54 blocks
==315== possibly lost: 3,528 bytes in 16 blocks
Will need to repeat this with the maplibre-gl-native bindings once those are more widely available.
Related https://github.com/brendan-ward/pymgl/issues/1
Possibly related: mapbox-gl-native #15915
It looks like some of the Node-related memory leaks were solved in 15922 but not actually merged; these may still need to be addressed in the latest maplibre-gl-native NodeJS bindings.