rspack
rspack copied to clipboard
[Bug]: Memory Leak in Dev Server Causing System to Run Out of Application Memory
Edited by @h-a-n-a:
For further issue reporting, please provide the system info. For example, Apple M2 Max, macOS 15.0.
Original Issue:
Version
System:
OS: macOS 14.6.1
CPU: (11) arm64 Apple M3 Pro
Memory: 119.66 MB / 18.00 GB
Shell: 5.9 - /bin/zsh
Browsers:
Safari: 17.6
npmPackages:
@rsbuild/[email protected]
@rsbuild/[email protected]
Details
While using rsbuild dev, I encountered a significant memory leak that caused the system to run out of application memory. This issue makes the system unstable and forces the user to quit applications to free up resources.
CodeSandbox: https://codesandbox.io/p/github/Fausto95/react-native-web-rsbuild/main?import=true
Reproduce link
https://github.com/Fausto95/react-native-web-rsbuild
Reproduce Steps
pnpm installpnpm start:web- Keep on applying changes to
/apps/rnw/src/App.js - Use
Activity Monitorand observe the continuous increase in memory usage for thersbuild-nodeprocess
I can report the same issue with rspack, rspack-node process can take ~30Gb of RAM.
I use @rspack/[email protected], rspack config can be found here
Not sure how to reproduce this amount of memory consumption, but I found there might be something related to memory reseting in mimalloc. I changed memory semantic from MADV_FREE to MADV_DONTNEED to reflect memory consumption accurately in statistics like RSS.
@h-a-n-a I'm ready to test possible fixes as just running dev server for a while increases memory consumption for me, it wasn't a problem in v1
@h-a-n-a I'm ready to test possible fixes as just running dev server for a while increases memory consumption for me, it wasn't a problem in v1
@uladzimirdev Do you mind adding a reproducible demo? Which version of rspack do you use that doesn't have this issue? You may also try @rspack-canary/[email protected]
@h-a-n-a
this is what I got on rspack serve
/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:14665
[js] this.#instance = new instanceBinding.Rspack(
[js] ^
[js]
[js] Error: Missing field `enforceRelative`
[js]
[js] at #getInstance (/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:14665:22)
[js] at #build (/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13879:22)
[js] at /node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13851:18
[js] at done (/node_modules/@rspack/lite-tapable/dist/index.js:473:13)
[js] at AsyncSeriesHook.callAsyncStageRange (/node_modules/@rspack/lite-tapable/dist/index.js:480:20)
[js] at AsyncSeriesHook.callAsync (/node_modules/@rspack/lite-tapable/dist/index.js:82:21)
[js] at _Compiler.compile (/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13845:30)
[js] at /node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13251:21
[js] at done (/node_modules/@rspack/lite-tapable/dist/index.js:473:13)
[js] at next (/metabase/node_modules/@rspack/lite-tapable/dist/index.js:533:25) {
[js] code: 'GenericFailure'
[js] }
[js]
[js] Node.js v18.20.5
error Command failed with exit code 1.
+1 leaking at macos intel.
this is what I got on
rspack serve/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:14665 [js] this.#instance = new instanceBinding.Rspack( [js] ^ [js] [js] Error: Missing field `enforceRelative` [js] [js] at #getInstance (/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:14665:22) [js] at #build (/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13879:22) [js] at /node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13851:18 [js] at done (/node_modules/@rspack/lite-tapable/dist/index.js:473:13) [js] at AsyncSeriesHook.callAsyncStageRange (/node_modules/@rspack/lite-tapable/dist/index.js:480:20) [js] at AsyncSeriesHook.callAsync (/node_modules/@rspack/lite-tapable/dist/index.js:82:21) [js] at _Compiler.compile (/node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13845:30) [js] at /node_modules/@rspack/cli/node_modules/@rspack/core/dist/index.js:13251:21 [js] at done (/node_modules/@rspack/lite-tapable/dist/index.js:473:13) [js] at next (/metabase/node_modules/@rspack/lite-tapable/dist/index.js:533:25) { [js] code: 'GenericFailure' [js] } [js] [js] Node.js v18.20.5 error Command failed with exit code 1.
@uladzimirdev Would you please try again with install-rspack?
@h-a-n-a I used it to install canary, let me try one more time.
yeah, no difference. It looks like it happens mostly when I switch branches without restarting dev server, memory grows with every HMR in action
Same here. Our application starts out with under 2GB of RAM usage for the rspack-node process, and easily accumulates around 20GB during usage. We are on 1.2.1 and i am using a Apple M3 Pro.
Are there any plans to investigate / fix this? Thank you for consideration.
Same for us, when starting rspack the memory usage is ±4GB and growth to around 50GB while developing. Using 1.2.1 with Apple M2 Pro
Hello
I'm having the same issue
System
OS: macOS 15.2
Chip: Apple M2 Pro
Memory: 16 GB
Dependencies
"@rsbuild/core": "^1.2.2",
"@rsbuild/plugin-babel": "^1.0.3",
"@rsbuild/plugin-react": "^1.1.0",
"@rsdoctor/rspack-plugin": "^0.4.13"
does it happen after upgrade to 1.2 @h-a-n-a can you take a look, seems some memory regression here
To resolve this issue as soon as possible, please provide as much context as possible:
- Does it occur when rspack was upgraded from A version to B version? If so, please provide these versions.
- Reproduction link and details of the steps to reproduce the issue.
- For the steps, does it includes any update to import/export statements or just general file edits?
cc @hnrchrdl @rafael-zilberman @xenokratos
Hello @h-a-n-a
I used the reproduce link from @Fausto95, and I have the same issue (even with @rsbuild/core: "^1.1.13")
Memory used on start-up
Memory used after some builds (at each save it increases a bit even without any change on the code)
After quitting and restarting the build
Memory used after some builds (Just by spamming cmd+s without any changes on the code)
Let me know if you need more context, or you want me to test things.
My team has seen it grow from 2GB up to 20GB+. It's present on at least 1.1 and 1.2; we have tested on both. It does appear to be HMR-related.
just general file edits
Yeah, it seems to be general edits. Not necessarily changes to imports.
The steps are as described:
- Keep on applying changes to (some file referenced in the build)
- Use Activity Monitor and observe the continuous increase in memory usage for the
rsbuild-nodeprocess
Are there any logs or dumps I can collect to help? Happy to jump on a call or share configs etc
Seeing this issue on the latest version (1.2.4). Will start at ~2GB and ramp up to 4-5GB over a few minutes without any file changes or recompilations.
Seemingly related: https://github.com/web-infra-dev/rsbuild/discussions/4455. I’m unable to increase the memory limit following the instructions in the docs: https://rsbuild.dev/guide/faq/exceptions#javascript-heap-out-of-memory-when-compiling. Appears to have no effect.
This, in combination with the ramp up in memory usage means my dev server will crash with JavaScript heap out of memory very quickly, requiring regular restarts.
system:
OS: macOS 14.4.1
CPU: Apple M2 Pro
Memory: 32GB
dependencies:
"@rsbuild/core": "1.2.4",
"@rsbuild/plugin-babel": "1.0.3",
"@rsbuild/plugin-css-minimizer": "1.0.2",
"@rsbuild/plugin-mdx": "1.0.2",
"@rsbuild/plugin-node-polyfill": "1.2.0",
"@rsbuild/plugin-react": "1.1.0",
"@rsbuild/plugin-type-check": "1.2.1",
downgraded rspack core and rspack cli to 1.0.1 and the issue persists, but not that significant. node process memory usage grew from 3.5Gb to 6.5Gb in an hour.
with 1.1.6 it grew from 3Gb to 45Gb
This might be related to memory cache.
I wrote an example. In this example, rspack was invoked rebuild for 4 times.
As you can see,
when memory cache was ON:
when memory cache was OFF:
This indicates that with memory cache turned on, the more rspack rebuilds, the more memory accumulates. Without memory cache, rspack rebuilds more slowly but also with lower memory retention. You can see every time rebuild was invoked, memory will drop down almost the same as it was when compiler was started.
Issue was created: https://github.com/web-infra-dev/rspack/issues/9405
This might be related to memory cache.
I wrote an example. In this example, rspack was invoked rebuild for 4 times.
As you can see,
when memory cache was ON:
when memory cache was **OFF**:
This indicates that with memory cache turned on, the more rspack rebuilds, the more memory accumulates. Without memory cache, rspack rebuilds more slowly but also with lower memory retention. You can see every time rebuild was invoked, memory will drop down almost the same as it was when compiler was started.
When I started debugging the app with HMR, after a few hours, rspack-node's memory usage reached 20G.
Now, I know what caused it. My question is, why didn't this problem occur in v1.0.0? What did you add later to cause this problem?
Is there a way to solve this problem (memory usage causes the computer to have no memory available, which is unacceptable)
@1yasa In prev versions, memory usage was lower based on my experiments
@h-a-n-a but with in memory cache disabled, every change causes full rebuild of the build
@1yasa:
Now, I know what caused it. My question is, why didn't this problem occur in v1.0.0? What did you add later to cause this problem?
On which version are you using now? I might need to do a bisect to see if anything caused this regression.
Is there a way to solve this problem (memory usage causes the computer to have no memory available, which is unacceptable)
I was thinking of clearing the memory cache if they are not necessary anymore.
@1yasa:
Now, I know what caused it. My question is, why didn't this problem occur in v1.0.0? What did you add later to cause this problem?
On which version are you using now? I might need to do a bisect to see if anything caused this regression.
Is there a way to solve this problem (memory usage causes the computer to have no memory available, which is unacceptable)
I was thinking of clearing the memory cache if they are not necessary anymore.
v1.1.8
@h-a-n-a Same problem for me
Reproduce link
https://github.com/memory1668/rsbuild-project-vue
This is a vue2 project created by pnpm create rsbuild@latest
Run pnpm dev, then run node test-memory-leak.js to trigger HMR every 2 seconds
Memory used on start-up
Memory used after some builds
@memory1668 Try turn off memory cache to see if memory is stable across builds.
@memory1668 Try turn off memory cache to see if memory is stable across builds.
Do you mean this config?
In fact, I just tried this configuration, but the memory growth was even faster
I got the reproduction running on Linux. The result showed the memory consumption is normal without cache enabled. If you happen to have a Linux, feel free to help me test it out your own project. That would be very helpful.
Update on the current status of this issue.
I've figured out there's some huge memory retention issue if cache was enabled during the build. This is not plaform specific. Here's some data on my ubuntu-22.04 when cache was enabled as a complement to my previous result on macOS using Instruments.app.
On top of that, we do have some platform specific issue related to macOS according to my investigation and this might be related to mimalloc, which is the memory allocator we used under the hood to speedup our compilation. I also filed an issue over there: https://github.com/microsoft/mimalloc/issues/1025. At the same time, I'm continuously trying to add a minimal reproducible demo to help tackle down the issue as soon as possible.
I've merged a fix to lift memory consumption issue of memory cache. Rspack will now clean the invalid memory cache. By default, memory cache unused in the next compilation will be removed (defaults maxGenerations is set to 1)https://github.com/web-infra-dev/rspack/pull/9517. I've tested it against a few hmr + rss memory tests, showing threejs_10x reducing almost 20% of memory usage. https://github.com/web-infra-dev/rspack/commit/25661fe9e41496966699e3427f19f97865746544#commitcomment-153133945
Took a test on mimalloc v3. Although memory is still growing, the issue has been greatly lifted. I've applied a fix to main and released a new canary if you would like to give it a shot right now, 1.2.8-canary-3d971bbf-20250304061156. Could use install-rspack to install canary. Refer to this PR to see changes in this fix https://github.com/web-infra-dev/rspack/pull/9533.
I'll hold off on closing this until it's fully fixed. It would be better if you could provide as much feedback as possible for this fix (better come up with the platform and OS info).
Thanks @h-a-n-a for working on this. I will try the canary and let you know how it goes.
when memory cache was **OFF**:
This indicates that with memory cache turned on, the more rspack rebuilds, the more memory accumulates. Without memory cache, rspack rebuilds more slowly but also with lower memory retention. You can see every time rebuild was invoked, memory will drop down almost the same as it was when compiler was started.