shadowsocks
memory leak on Node v0.10
The newest version 0.10.2 works fine with me. 0.6.0 is not. with 0.6.0, CPU usage shoots to 100% and the thread exits unexpectedly quite often.
Might this be the reason for the mem leak?
In versions of node prior to v0.10, the incoming message data would be simply discarded. However, in Node v0.10 and beyond, the socket will remain paused forever.
More info is at http://nodejs.org/api/stream.html#stream_compatibility
node v0.10.4 shadowsocks 0.10 dev Received an warning message when running. But it continues to work fine.
(node) warning: possible EventEmitter memory leak detected. 11 listeners added. Use emitter.setMaxListeners() to increase limit.
Trace
at Socket.EventEmitter.addListener (events.js:160:15)
at Socket.Readable.on (_stream_readable.js:653:33)
at Socket.EventEmitter.once (events.js:179:8)
at Socket.destroySoon (net.js:413:10)
at Socket.onSocketEnd (net.js:261:10)
at Socket.EventEmitter.emit (events.js:92:17)
at TCP.onread (net.js:535:10)
Used as proxy for xunlei for several hours and memory footprint starts to go up.
$ ps aux|grep node
1000 29425 0.0 28.5 353452 292848 ? Sl Jun03 5:20 node server.js
$ node -v
v0.10.2
I'll check tomorrow after timeout 600s went out.
Forcing gc() every few seconds seems help. c4ca46610d5cfa335aea2789508ce620bbede539
Other possible causes:
- pause() is not working in 0.10
- write() buffer is too large?
It seems that inbound traffic is always a bit higher than outbound traffic.
Found a problem with crypto module https://github.com/joyent/node/issues/5949
Is this fixed on upgrade to 0.12 or iojs? From this "We have the new TLSWrap mechanism under the hood, this eliminates quite a few of the hops back and forth between JavaScript and our C++ implementations." so this would make the nodejs port at least faster I'd guess.
IMHO this is not a leak, at least what you've linked on https://github.com/joyent/node/issues/5949. Despite this comment from one of top node hackers, from my own experience such greedy memory consumption is common also to Java. I guess node will do better GC job when system is really under memory pressure but otherwise, it will eat as much as it can, just like what Java do for decreasing overhead of OS malloc.
@ebraminio I have already abandoned this project. No matter this is a leak or not, I can not solve it. I have been debugging node and V8 source for several weeks without any progress.
Now I'm happy with Shadowsocks Python project. I can handle 10K connections with < 100MB RSS without having to think about GC.
I'm working on a new version of shdowsocks-gui now. But I haven't touch shadowsocks-nodejs yet. If there's anybody who want to fork this project and keep working on it, I'd like integrate it into my shadowsocks-gui fork.
To complete the story, this is likely a backpressure related issue in earlier version of Streams, readable.pipe and stream.pipeline has been provided since v9 and v10 to free user from manual labor, and it's quite reliable nowadays.
node_args: ["--max_old_space_size=64", "--max_semi_space_size=2"],
env: {
"LD_PRELOAD": "/usr/lib/x86_64-linux-gnu/libjemalloc.so.2"
},
Using this parameter solves the memory problem.
