eslint_d.js icon indicating copy to clipboard operation
eslint_d.js copied to clipboard

Memory used by background process increases endlessly.

Open mteplyi opened this issue 1 year ago • 5 comments

Hi, it is awesome project, but I noticed probably a memory leak. If server is already running, I change any file from check_files array (package.json, package-lock.json, npm-shrinkwrap.json, yarn.lock, pnpm-lock.yaml) and run eslint_d again, then the memory used by background eslint_d process increases for about 100-200 MB. It repeats over and over again. For me it reached even 3 GB and then I have to restart it manually. I run eslint_d for the first time and every next time with the same command and only in one project: eslint_d --cache --fix **/*.ts I have spent a lot of time trying to manage this issue in source code, but I can not understand where exactly it happens. Can someone help with it?

mteplyi avatar Oct 02 '23 13:10 mteplyi

Seems like a lot of stuff was not deleted from memory on clear before new cache creation. I tried to analyze a heap and can assume, that it hangs somewhere in ESLint class and so deletion of require.cache and lru entries does not kill all refs. Maybe, it will be simpler to just restart background eslint_d process on those check_files changes..

And also it would be great to have an ability to pass such check_files through cli args.

mteplyi avatar Oct 02 '23 19:10 mteplyi

image

The same issue if run over a directory of files without --cache flag. With --cache it doesn't take more than 200mb of RAM. Probably worth to enable this flag by default.

To reproduce, clone this repo: AykutSarac/jsoncrack.com

eslint_d ./
❯  eslint_d -v
v8.52.0 (eslint_d v13.0.0)

UPD: The issue is rather occasional

hinell avatar Oct 30 '23 07:10 hinell

Can confirm. I'm getting over 1.2GB of memory usage

frixaco avatar Nov 12 '23 19:11 frixaco

My best guess is that you have plugins that use import instead of require. While eslint_d clears the require cache, there is no solution for modules in place:

https://github.com/mantoni/eslint_d.js/blob/8357a5929dae7d2979680069107da308eab0a450/lib/caches.js#L61-L67

Does anyone knows if/how imported modules can be cleared? Would we need a custom loader?

An alternative would be to change the architecture of eslint_d and launch a separate process for each project (cwd). When reloading the project, we could dispose the process and re-create it. Any thoughts?

mantoni avatar Nov 13 '23 08:11 mantoni

Screenshot 2024-01-05 at 2 34 52 PM

I'm using Neovim and I set up eslint_d through Mason: https://github.com/williamboman/mason.nvim

Sitting at 4GB memory consumption. And I have no instance of nvim open currently.

Amar-Gill avatar Jan 05 '24 19:01 Amar-Gill

This is probably obsolete with the new major v14.0.0 (see #294). Should you're still having the issue, please write here and I'm happy to re-open.

mantoni avatar Jul 28 '24 15:07 mantoni