luacheck Memory usage increase

We have observed that running luacheck over all Lua files in Prosody (current-ish trunk) hits a peak RSS of around 100M RSS.

Using bisect to narrow it down, these commits stood out as major jumps in memory usage (20-25M increase wrt the previous):

https://github.com/mpeterv/luacheck/commit/9bdd7754dab71351de72da461cdf2cafcee9d212
https://github.com/mpeterv/luacheck/commit/745673a105419dbe544ef6063791953cd72680fc

Jun 19 '18 15:06 Zash

That's a lot! It seems that the warning filtering stage tries to avoid mutating tables too hard, by creating a lot of fresh objects, and these commits increased amount of various filtering done. This situation can definitely be improved, I'll try to look into it soon.

Jun 19 '18 20:06 mpeterv

After some improvements on master branch peak RSS went down from ~92MB to ~68MB (running luacheck . with Lua 5.1 and --cache enabled and fully hit). I'll keep working on this and make a patch release in a week or so.

Jun 21 '18 15:06 mpeterv

0.22.1 released, peak RSS down to ~55MB. I'll keep working on this, some optimizations require slightly changing behavior in some corner cases so they'll go into 0.23.0.

Jul 01 '18 17:07 mpeterv

Thanks! Will try.

FWIW, the backstory here was that we were playing with CI test runners on a virtual machine with 1GB RAM, with about half of that in use by other things. We had 4 different test environments, which used about 20M each when idle. So you can imagine how much was left when a test run was triggered and it ran 4 instances of luacheck at the same time.

So, this helps a lot. We'll also be looking into smarter scheduling and isolating things from other more important services.

Jul 01 '18 20:07 Zash