Memory leak somewhere, bot creating 40 thousand string objects

[jasonwilliams]

I've been inspecting with heapdump (i can attach the file if people need) http://imgur.com/a/iCuqA

[jasonwilliams]

pings @ljharb @gkatsev

[dsamarin]

Hm. Very strange. I don't recognize any of those strings but they look like they relate to libraries that Node.js is including. What method are you using to generate and view the heapdump? Also, is there a way to check how the memory usage increases over time?

[jasonwilliams]

@dsamarin I have just pulled the latest version of this bot down, so i will show an example of me using heapdump again. I've basically made a user command that dumps it, so i can make a new dump whenever, i've left the bot running for a day, then made a second dump, and compared the two.

The new bot is still memory leaking to a high degree. unfortunately that heapdump isn't helping me locate where the problem is happening :/ yes i do think its a low level library, but im not sure which one

[jasonwilliams]

@dsamarin @ljharb Steps to replicate

• add heapdump to the package.json
• go into your-bot-here.js, and add heapdump as a module at the top
• inside the ping command, add a heapdump.writesnapshot() function call

see this branch... I've included the heapdump (which was taken around 5 minutes after the bot running) https://github.com/Jayflux/oftn-bot/tree/heap_debug

• Go in to my branch
• download the heapsnapshot
• open up chrome dev tools, navigate to profile, and load the snapshot
• after 5 mins of running there's already 4,941 string objects in memory

Although this doesn't seem much, after leaving the bot running for a day or 2, its memory usage goes up to 20% of the whole system, and ends up being killed by the kernal

[jasonwilliams]

ended up using node-inspector instead, looks like some of those strings are coming from the node core itself

[yorickvP]

The heapdump you included doesn't seem all that terrible (most of them are code / function names). Might be useful if you could take diff's somehow.

[jasonwilliams]

@yorickvP I will try and get you some diffs Also, not sure if this helps but this is Ella after 5 hours of running https://github.com/hashweb/ella/blob/master/ella.js

   PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
 4269 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.04 node /home/ella/ella/ella.js 
 4270 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.05 node /home/ella/ella/ella.js 
 4271 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.04 node /home/ella/ella/ella.js 
 4272 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.04 node /home/ella/ella/ella.js 
 4273 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.00 node /home/ella/ella/ella.js 
 4274 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.00 node /home/ella/ella/ella.js 
 4275 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.00 node /home/ella/ella/ella.js 
 4276 ella       20   0 1250M 77592 15984 S  0.0  7.6  0:00.00 node /home/ella/ella/ella.js 

7.6 is quite high, considering i have a bot in python which is more huge (https://github.com/ProgVal/Limnoria) writing to the DB, generating logs etc and only using 2.8 this is after a couple of days running

PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
22491 logbot     20   0  157M 28640  8576 S  0.0  2.8  3:14.01 │              └─ /usr/bin/python3 /usr/local/bin/supybot __Lauren.conf