jest [Bug]: Memory consumption issues on Node JS 16.11.0+

[Bug]: Memory consumption issues on Node JS 16.11.0+

Open EternallLight opened this issue 2 years ago • 172 comments

🚨 Using Jest 29, you can use --workerIdleMemoryLimit. See https://jestjs.io/docs/configuration/#workeridlememorylimit-numberstring 🚨

Version

27.0.6

Steps to reproduce

Install the latest Node JS (16.11.0 or later) or use the appropriate Docker image
Set up a project with a multiplicity Jest tests
Run node --expose-gc node_modules/.bin/jest --logHeapUsage and see how the memory consumption starts increasing.

Expected behavior

Since Jest calls global.gc() when Garbage Collector is exposed and --logHeapUsage flag is present, the memory usage should be stable.

Actual behavior

The memory usage increases with every new test

Additional context

We had some issues with Jest workers consuming all available RAM both on CI machine and locally. After doing some research, we found that if we run Jest like the following node --expose-gc node_modules/.bin/jest --logHeapUsage, the heap size remains stable. After upgrading to Node JS v16.11.0, the issue was back. Node v16.10.0 works fine. I believe it was something accidentally introduced in the new Node, but it might be useful to take a look at this from Jest perspective in search of possible workarounds. I'm also having the same behavior on my working machine, environment of which I'm pasting below 👇🏻

Environment

System:
    OS: macOS 11.6
    CPU: (8) x64 Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
  Binaries:
    Node: 16.11.0 - ~/.nvm/versions/node/v16.11.0/bin/node
    Yarn: 1.22.0 - ~/SomeFancyDir/webapp/node_modules/.bin/yarn
    npm: 8.0.0 - ~/.nvm/versions/node/v16.11.0/bin/npm
  npmPackages:
    jest: 27.0.6 => 27.0.6

Oct 13 '21 09:10 EternallLight

We're also experiencing this issue. Node 16.11+ and Jest v27 consumes significantly more memory. Node 16.10 and Jest v27 seems OK.

Nov 09 '21 13:11 rthreei

We're also unable to update to the LTS version of Node 16 (at the time of writing 16.13.0) because of this issue. We bisected the changes and identified that the upgrade from Node 16.10 to 16.11 caused our large Jest suite to hang indefinitely.

I took a look at the Node 16.11 changelog and I think the most likely culprit for this issue comes from the V8 update to 9.4 (PR). In V8 9.4, the new Sparkplug compiler is enabled by default (see also this Node issue).

I was hoping I could try disabling sparkplug to see verify that this is the issue. node exposes a V8 option to disable (--no-sparkplug), but I don't think it's passing through to the Jest workers when I call it like this:

node --no-sparkplug node_modules/.bin/jest

I also tried setting the V8 option in jest-environment-node here: https://github.com/facebook/jest/blob/42b020f2931ac04820521cc8037b7c430eb2fa2f/packages/jest-environment-node/src/index.ts#L109 via

setFlagsFromString('--no-sparkplug');

but I didn't see any change. I'm not sure if that means Sparkplug isn't causing the problem or if I'm not setting the V8 flag properly in the jest workers.

@SimenB - I see you've committed a good deal to jest-environment-node - any tips for how I might pass that V8 flag down through all the workers? If it's possible (even via patch-pacakge or something) I'd be happy to give it a shot on our test suite that's exhibiting the problem.

So I'm not exactly positive that this is the cause of the issue, but it seems like a potentially promising place to start looking.

Nov 12 '21 17:11 blimmer

any tips for how I might pass that V8 flag down through all the workers?

I would have thought https://github.com/facebook/jest/blob/e0b33b74b5afd738edc183858b5c34053cfc26dd/packages/jest-worker/src/workers/ChildProcessWorker.ts#L93-L94 made it so it was passed down...

Nov 12 '21 17:11 SimenB

It's possible the sparkplug thing is a red herring. Nothing else jumped out at me from that changelog.

I captured some heap size metrics to show the scale of the difference.

Node 16.10.0 (baseline)

> node --expose-gc ./node_modules/.bin/jest --logHeapUsage --maxWorkers ${TEST_PARALLELISM:-8} "--reporters=default" "--reporters=jest-junit"
(14.297 s, 211 MB heap size)
(16.811 s, 211 MB heap size)
(17.913 s, 211 MB heap size)
(18.382 s, 213 MB heap size)
(18.38 s, 214 MB heap size)
(19.122 s, 238 MB heap size)
(19.241 s, 211 MB heap size)
(21.969 s, 222 MB heap size)
(10.783 s, 268 MB heap size)
(6.974 s, 268 MB heap size)
(7.575 s, 267 MB heap size)
(7.404 s, 269 MB heap size)
(6.788 s, 288 MB heap size)
(6.907 s, 267 MB heap size)
(10.324 s, 268 MB heap size)
(6.796 s, 276 MB heap size)
(5.177 s, 282 MB heap size)
(5.638 s, 269 MB heap size)
(5.702 s, 269 MB heap size)
(268 MB heap size)
(6.432 s, 271 MB heap size)
(7.227 s, 268 MB heap size)
(7.276 s, 272 MB heap size)
(5.192 s, 278 MB heap size)
(271 MB heap size)
(5.053 s, 286 MB heap size)
(269 MB heap size)
(5.3 s, 272 MB heap size)
(281 MB heap size)
(274 MB heap size)
(281 MB heap size)
(5.018 s, 272 MB heap size)
(274 MB heap size)
(288 MB heap size)
(275 MB heap size)
(272 MB heap size)
(276 MB heap size)
(278 MB heap size)
(285 MB heap size)
(274 MB heap size)
(290 MB heap size)
(277 MB heap size)
(275 MB heap size)
(278 MB heap size)
(278 MB heap size)
(280 MB heap size)
(287 MB heap size)
(277 MB heap size)
(293 MB heap size)
(280 MB heap size)
(277 MB heap size)
(281 MB heap size)
(281 MB heap size)
(280 MB heap size)
(283 MB heap size)
(289 MB heap size)
(296 MB heap size)
(283 MB heap size)
(280 MB heap size)
(284 MB heap size)
(283 MB heap size)
(282 MB heap size)
(285 MB heap size)
(292 MB heap size)
(298 MB heap size)
(282 MB heap size)
(285 MB heap size)
(286 MB heap size)
(286 MB heap size)
(285 MB heap size)
(287 MB heap size)
(295 MB heap size)
(301 MB heap size)
(285 MB heap size)
(288 MB heap size)
(289 MB heap size)
(289 MB heap size)
(288 MB heap size)
(290 MB heap size)
(298 MB heap size)
(216 MB heap size)
(304 MB heap size)
(288 MB heap size)
(290 MB heap size)
(292 MB heap size)
(292 MB heap size)
(296 MB heap size)
(294 MB heap size)
(295 MB heap size)
(293 MB heap size)
(6.766 s, 283 MB heap size)
(7.887 s, 273 MB heap size)
(7.654 s, 308 MB heap size)
(8.406 s, 272 MB heap size)
(298 MB heap size)
(298 MB heap size)
(297 MB heap size)
(295 MB heap size)
(297 MB heap size)
(300 MB heap size)
(289 MB heap size)
(305 MB heap size)
(301 MB heap size)
(300 MB heap size)
(291 MB heap size)
(297 MB heap size)
(300 MB heap size)
(303 MB heap size)
(303 MB heap size)
(292 MB heap size)
(303 MB heap size)
(308 MB heap size)
(311 MB heap size)
(300 MB heap size)
(303 MB heap size)
(306 MB heap size)
(295 MB heap size)
(306 MB heap size)
(310 MB heap size)
(305 MB heap size)
(297 MB heap size)
(303 MB heap size)
(306 MB heap size)
(309 MB heap size)
(297 MB heap size)
(313 MB heap size)
(310 MB heap size)
(307 MB heap size)
(305 MB heap size)
(299 MB heap size)
(308 MB heap size)
(311 MB heap size)
(315 MB heap size)
(300 MB heap size)
(311 MB heap size)
(220 MB heap size)
(208 MB heap size)
(310 MB heap size)
(302 MB heap size)
(309 MB heap size)
(242 MB heap size)
(311 MB heap size)
(284 MB heap size)
(322 MB heap size)
(296 MB heap size)
(312 MB heap size)
(313 MB heap size)
(315 MB heap size)
(317 MB heap size)
(316 MB heap size)
(304 MB heap size)
(315 MB heap size)
(308 MB heap size)
(315 MB heap size)
(317 MB heap size)
(318 MB heap size)
(317 MB heap size)
(320 MB heap size)
(306 MB heap size)
(311 MB heap size)
(317 MB heap size)
(320 MB heap size)
(318 MB heap size)
(320 MB heap size)
(323 MB heap size)
(320 MB heap size)
(310 MB heap size)
(313 MB heap size)
(323 MB heap size)
(320 MB heap size)
(321 MB heap size)
(340 MB heap size)
(326 MB heap size)
(323 MB heap size)
(312 MB heap size)
(326 MB heap size)
(316 MB heap size)
(322 MB heap size)
(328 MB heap size)
(323 MB heap size)
(326 MB heap size)
(343 MB heap size)
(315 MB heap size)
(329 MB heap size)
(325 MB heap size)
(319 MB heap size)
(328 MB heap size)
(326 MB heap size)
(331 MB heap size)
(345 MB heap size)
(318 MB heap size)
(331 MB heap size)
(328 MB heap size)
(321 MB heap size)
(329 MB heap size)
(332 MB heap size)
(334 MB heap size)
(348 MB heap size)
(320 MB heap size)
(334 MB heap size)
(331 MB heap size)
(324 MB heap size)
(335 MB heap size)
(332 MB heap size)
(337 MB heap size)
(350 MB heap size)
(323 MB heap size)
(265 MB heap size)
(333 MB heap size)
(337 MB heap size)
(302 MB heap size)
(240 MB heap size)
(327 MB heap size)
(257 MB heap size)
(266 MB heap size)
(334 MB heap size)
(337 MB heap size)
(9.083 s, 342 MB heap size)
(9.235 s, 319 MB heap size)
(9.967 s, 321 MB heap size)
(9.455 s, 345 MB heap size)
(9.686 s, 359 MB heap size)
(12.938 s, 313 MB heap size)
(13.969 s, 340 MB heap size)
(14.6 s, 333 MB heap size)
(6.149 s, 340 MB heap size)
(8.209 s, 347 MB heap size)
(9.345 s, 337 MB heap size)
(10 s, 344 MB heap size)
(8.202 s, 360 MB heap size)
(7.368 s, 333 MB heap size)
(6.233 s, 342 MB heap size)
(6.243 s, 359 MB heap size)
(6.108 s, 342 MB heap size)
(363 MB heap size)
(5.324 s, 315 MB heap size)
(332 MB heap size)
(6.147 s, 338 MB heap size)
(345 MB heap size)
(5.909 s, 341 MB heap size)
(7.36 s, 325 MB heap size)
(5.155 s, 342 MB heap size)
(324 MB heap size)
(5.058 s, 364 MB heap size)
(313 MB heap size)
(5.34 s, 276 MB heap size)
(273 MB heap size)
(342 MB heap size)
(281 MB heap size)
(271 MB heap size)
(327 MB heap size)
(273 MB heap size)
(344 MB heap size)
(276 MB heap size)
(296 MB heap size)
(273 MB heap size)
(281 MB heap size)
(271 MB heap size)
(276 MB heap size)
(289 MB heap size)
(299 MB heap size)
(347 MB heap size)
(277 MB heap size)
(275 MB heap size)
(283 MB heap size)
(274 MB heap size)
(279 MB heap size)
(291 MB heap size)
(350 MB heap size)
(302 MB heap size)
(279 MB heap size)
(278 MB heap size)
(286 MB heap size)
(277 MB heap size)
(281 MB heap size)
(294 MB heap size)
(353 MB heap size)
(283 MB heap size)
(284 MB heap size)
(289 MB heap size)
(281 MB heap size)
(281 MB heap size)
(285 MB heap size)
(297 MB heap size)
(285 MB heap size)
(355 MB heap size)
(287 MB heap size)
(283 MB heap size)
(292 MB heap size)
(282 MB heap size)
(287 MB heap size)
(300 MB heap size)
(286 MB heap size)
(358 MB heap size)
(287 MB heap size)
(294 MB heap size)
(290 MB heap size)
(285 MB heap size)
(290 MB heap size)
(302 MB heap size)
(289 MB heap size)
(297 MB heap size)
(290 MB heap size)
(292 MB heap size)
(361 MB heap size)
(365 MB heap size)
(5.125 s, 298 MB heap size)
(6.13 s, 295 MB heap size)
(6.526 s, 304 MB heap size)
(6.928 s, 301 MB heap size)
(7.982 s, 281 MB heap size)
(8.516 s, 317 MB heap size)
(9.841 s, 313 MB heap size)
(5.477 s, 295 MB heap size)
(5.506 s, 312 MB heap size)
(296 MB heap size)
(294 MB heap size)
(301 MB heap size)
(298 MB heap size)
(326 MB heap size)
(301 MB heap size)
(297 MB heap size)
(294 MB heap size)
(294 MB heap size)
(330 MB heap size)
(295 MB heap size)
(301 MB heap size)
(330 MB heap size)
(296 MB heap size)
(323 MB heap size)
(297 MB heap size)
(297 MB heap size)
(316 MB heap size)
(297 MB heap size)
(304 MB heap size)
(332 MB heap size)
(298 MB heap size)
(308 MB heap size)
(319 MB heap size)
(300 MB heap size)
(300 MB heap size)
(300 MB heap size)
(334 MB heap size)
(306 MB heap size)
(301 MB heap size)
(310 MB heap size)
(303 MB heap size)
(322 MB heap size)
(302 MB heap size)
(302 MB heap size)
(337 MB heap size)
(310 MB heap size)
(212 MB heap size)
(209 MB heap size)
(304 MB heap size)
(253 MB heap size)
(230 MB heap size)
(265 MB heap size)
(286 MB heap size)
(312 MB heap size)
(5.316 s, 317 MB heap size)
(5.007 s, 307 MB heap size)
(8.481 s, 309 MB heap size)
(8.669 s, 289 MB heap size)
(10.421 s, 313 MB heap size)
(283 MB heap size)
(5.801 s, 316 MB heap size)
(336 MB heap size)
(13.269 s, 344 MB heap size)
(288 MB heap size)
(14.837 s, 338 MB heap size)
(288 MB heap size)
(331 MB heap size)
(285 MB heap size)
(302 MB heap size)
(287 MB heap size)
(340 MB heap size)
(297 MB heap size)
(297 MB heap size)
(287 MB heap size)
(313 MB heap size)
(291 MB heap size)
(343 MB heap size)
(300 MB heap size)
(279 MB heap size)
(298 MB heap size)
(290 MB heap size)
(293 MB heap size)
(314 MB heap size)
(303 MB heap size)
(345 MB heap size)
(282 MB heap size)
(301 MB heap size)
(25.518 s, 290 MB heap size)
(293 MB heap size)
(295 MB heap size)
(317 MB heap size)
(305 MB heap size)
(284 MB heap size)
(348 MB heap size)
(303 MB heap size)
(295 MB heap size)
(298 MB heap size)
(320 MB heap size)
(306 MB heap size)
(308 MB heap size)
(287 MB heap size)
(351 MB heap size)
(306 MB heap size)
(298 MB heap size)
(301 MB heap size)
(322 MB heap size)
(310 MB heap size)
(309 MB heap size)
(290 MB heap size)
(353 MB heap size)
(308 MB heap size)
(300 MB heap size)
(264 MB heap size)
(303 MB heap size)
(312 MB heap size)
(326 MB heap size)
(292 MB heap size)
(313 MB heap size)
(268 MB heap size)
(356 MB heap size)
(233 MB heap size)
(220 MB heap size)
(254 MB heap size)
(252 MB heap size)
(232 MB heap size)
(255 MB heap size)
(232 MB heap size)

Node 16.11.0 (problem introduced)

> node --expose-gc --no-sparkplug ./node_modules/.bin/jest --logHeapUsage --maxWorkers ${TEST_PARALLELISM:-8} "--reporters=default" "--reporters=jest-junit"
(41.727 s, 217 MB heap size)
(42.441 s, 217 MB heap size)
(42.452 s, 216 MB heap size)
(45.134 s, 217 MB heap size)
(45.868 s, 220 MB heap size)
(46.713 s, 245 MB heap size)
(49.371 s, 247 MB heap size)
(9.725 s, 328 MB heap size)
(12.574 s, 329 MB heap size)
(54.825 s, 218 MB heap size)
(8.787 s, 340 MB heap size)
(13.189 s, 329 MB heap size)
(13.471 s, 328 MB heap size)
(10.219 s, 341 MB heap size)
(14.752 s, 333 MB heap size)
(8.109 s, 397 MB heap size)
(10.887 s, 386 MB heap size)
(13.786 s, 387 MB heap size)
(8.03 s, 385 MB heap size)
(13.016 s, 388 MB heap size)
(9.036 s, 419 MB heap size)
(332 MB heap size)
(16.237 s, 328 MB heap size)
(10.056 s, 390 MB heap size)
(7.183 s, 458 MB heap size)
(6.975 s, 446 MB heap size)
(9.844 s, 411 MB heap size)
(7.222 s, 423 MB heap size)
(7.276 s, 426 MB heap size)
(6.49 s, 520 MB heap size)
(7.583 s, 444 MB heap size)
(14.041 s, 416 MB heap size)
(10.527 s, 386 MB heap size)
(6.191 s, 415 MB heap size)
(8.422 s, 510 MB heap size)
(6.528 s, 581 MB heap size)
(7.28 s, 486 MB heap size)
(9.558 s, 523 MB heap size)
(7.271 s, 423 MB heap size)
(6.011 s, 545 MB heap size)
(7.171 s, 477 MB heap size)
(9.289 s, 414 MB heap size)
(16.893 s, 496 MB heap size)
(9.017 s, 642 MB heap size)
(7.173 s, 521 MB heap size)
(9.823 s, 548 MB heap size)
(6.973 s, 606 MB heap size)
(7.009 s, 565 MB heap size)
(6.953 s, 416 MB heap size)
(8.185 s, 483 MB heap size)
(581 MB heap size)
(8.146 s, 489 MB heap size)
(5.061 s, 609 MB heap size)
(5.397 s, 666 MB heap size)
(6.634 s, 541 MB heap size)
(5.866 s, 476 MB heap size)
(8.275 s, 670 MB heap size)
(529 MB heap size)
(9.576 s, 478 MB heap size)
(6.206 s, 643 MB heap size)
(7.119 s, 670 MB heap size)
(6.208 s, 538 MB heap size)
(10.041 s, 550 MB heap size)
(6.383 s, 623 MB heap size)
(8.896 s, 727 MB heap size)
(6.355 s, 477 MB heap size)
(6.294 s, 704 MB heap size)
(5.112 s, 599 MB heap size)
(12.077 s, 748 MB heap size)
(5.699 s, 731 MB heap size)
(730 MB heap size)
(6.233 s, 722 MB heap size)
(7.113 s, 549 MB heap size)
(8.579 s, 684 MB heap size)
(6.936 s, 776 MB heap size)
(6.933 s, 659 MB heap size)
(6.843 s, 794 MB heap size)
(10.345 s, 539 MB heap size)
(5.048 s, 842 MB heap size)
(609 MB heap size)
(5.361 s, 783 MB heap size)
(6.001 s, 682 MB heap size)
(6.456 s, 828 MB heap size)
(5.785 s, 854 MB heap size)
(5.447 s, 451 MB heap size)
(5.668 s, 670 MB heap size)
(5.305 s, 843 MB heap size)
(7.549 s, 720 MB heap size)
(6.11 s, 903 MB heap size)
(5.761 s, 743 MB heap size)
(731 MB heap size)
(6.164 s, 889 MB heap size)
(5.513 s, 914 MB heap size)
(5.31 s, 904 MB heap size)
(5.111 s, 963 MB heap size)
(6.897 s, 512 MB heap size)
(6.214 s, 782 MB heap size)
(5.568 s, 805 MB heap size)
(573 MB heap size)
(5.445 s, 949 MB heap size)
(1024 MB heap size)
(6.722 s, 793 MB heap size)
(842 MB heap size)
(6.405 s, 964 MB heap size)
(8.361 s, 975 MB heap size)
(5.613 s, 865 MB heap size)
(902 MB heap size)
(5.719 s, 1009 MB heap size)
(5.861 s, 1084 MB heap size)
(5.734 s, 854 MB heap size)
(5.225 s, 1026 MB heap size)
(7.716 s, 633 MB heap size)
(5.191 s, 926 MB heap size)
(6.251 s, 974 MB heap size)
(1069 MB heap size)
(1085 MB heap size)
(1145 MB heap size)
(6.744 s, 962 MB heap size)
(5.617 s, 915 MB heap size)
(6.088 s, 653 MB heap size)
(1034 MB heap size)
(5.324 s, 987 MB heap size)
(5.146 s, 1130 MB heap size)
(1146 MB heap size)
(713 MB heap size)
(5.612 s, 1223 MB heap size)
(5.552 s, 975 MB heap size)
(1094 MB heap size)
(6.886 s, 963 MB heap size)
(6.49 s, 1048 MB heap size)
(1205 MB heap size)
(5.253 s, 1192 MB heap size)
(5.241 s, 775 MB heap size)
(1265 MB heap size)
(1156 MB heap size)
(1022 MB heap size)
(6.167 s, 1035 MB heap size)
(5.235 s, 1266 MB heap size)
(5.017 s, 1273 MB heap size)
(7.344 s, 1107 MB heap size)
(5.872 s, 1329 MB heap size)
(5.267 s, 1084 MB heap size)
(6.312 s, 1216 MB heap size)
(9.271 s, 837 MB heap size)
(1329 MB heap size)
(1315 MB heap size)
(10.02 s, 1107 MB heap size)
(1389 MB heap size)
(6.675 s, 983 MB heap size)
(1144 MB heap size)
(1276 MB heap size)
(5.482 s, 775 MB heap size)
(1375 MB heap size)
(5.25 s, 1390 MB heap size)
(5.264 s, 1204 MB heap size)
(5.69 s, 1449 MB heap size)
(5.953 s, 1042 MB heap size)
(6.997 s, 1031 MB heap size)
(1339 MB heap size)
(6.208 s, 845 MB heap size)
(5.602 s, 1435 MB heap size)
(1265 MB heap size)
(1092 MB heap size)
(6.002 s, 1103 MB heap size)
(898 MB heap size)
(9.446 s, 1462 MB heap size)
(6.544 s, 1510 MB heap size)
(6.159 s, 1400 MB heap size)
(1495 MB heap size)
(1328 MB heap size)
(5.141 s, 1152 MB heap size)
(1513 MB heap size)
(5.133 s, 957 MB heap size)
(5.965 s, 1164 MB heap size)
(1571 MB heap size)
(6.002 s, 1397 MB heap size)
(5.063 s, 1556 MB heap size)
(5.68 s, 1388 MB heap size)
(1018 MB heap size)
(6.518 s, 1212 MB heap size)
(5.141 s, 1631 MB heap size)
(6.407 s, 1574 MB heap size)
(6.51 s, 1224 MB heap size)
(5.082 s, 1466 MB heap size)
(1616 MB heap size)
(1078 MB heap size)
(1448 MB heap size)
(6.306 s, 1273 MB heap size)
(5.142 s, 1691 MB heap size)
(1677 MB heap size)
(5.811 s, 1077 MB heap size)
(1139 MB heap size)
(6.226 s, 1435 MB heap size)
(1508 MB heap size)
(1138 MB heap size)
(6.122 s, 1213 MB heap size)
(1736 MB heap size)
(1199 MB heap size)
(1496 MB heap size)
(15.376 s, 1528 MB heap size)
(1272 MB heap size)
(5.026 s, 1198 MB heap size)
(10.881 s, 1763 MB heap size)
(1258 MB heap size)
(8.77 s, 1580 MB heap size)
(1556 MB heap size)
(5.052 s, 1797 MB heap size)
(1586 MB heap size)
(1333 MB heap size)
(1322 MB heap size)
(1631 MB heap size)
(5.567 s, 1259 MB heap size)
(1616 MB heap size)
(5.411 s, 1857 MB heap size)
(6.565 s, 1755 MB heap size)
(1646 MB heap size)
(1691 MB heap size)
(1322 MB heap size)
(5.522 s, 1382 MB heap size)
(5.783 s, 1393 MB heap size)
(1549 MB heap size)
(5.275 s, 1676 MB heap size)
(1918 MB heap size)
(1442 MB heap size)
(1751 MB heap size)
(5.368 s, 1382 MB heap size)
(5.263 s, 1454 MB heap size)
(1661 MB heap size)
(5.433 s, 1736 MB heap size)
(1978 MB heap size)
(1502 MB heap size)
(1442 MB heap size)
(1812 MB heap size)
(1514 MB heap size)
(1737 MB heap size)
(1796 MB heap size)
(2038 MB heap size)
(17.555 s, 1800 MB heap size)
(1563 MB heap size)
(1502 MB heap size)
(1872 MB heap size)
(1574 MB heap size)
(1797 MB heap size)
(1428 MB heap size)
(1857 MB heap size)
(2099 MB heap size)
(1750 MB heap size)
(1562 MB heap size)
(6.206 s, 1569 MB heap size)
(1635 MB heap size)
(1933 MB heap size)
(6.031 s, 1560 MB heap size)
(1917 MB heap size)
(1862 MB heap size)
(2159 MB heap size)
(1623 MB heap size)
(1629 MB heap size)
(1694 MB heap size)
(5.348 s, 1637 MB heap size)
(1938 MB heap size)
(2218 MB heap size)
(6.073 s, 1978 MB heap size)
(1689 MB heap size)
(1683 MB heap size)
(1755 MB heap size)
(1543 MB heap size)
(1697 MB heap size)
(1998 MB heap size)
(2038 MB heap size)
(5.306 s, 2285 MB heap size)
(1749 MB heap size)
(1815 MB heap size)
(1677 MB heap size)
(2072 MB heap size)
(1757 MB heap size)
(2059 MB heap size)
(2098 MB heap size)
(2345 MB heap size)
(1875 MB heap size)
(1753 MB heap size)
(1817 MB heap size)
(2115 MB heap size)
(8.092 s, 1810 MB heap size)
(2118 MB heap size)
(2158 MB heap size)
(2405 MB heap size)
(1936 MB heap size)
(1814 MB heap size)
(5.142 s, 1877 MB heap size)
(1869 MB heap size)
(2138 MB heap size)
(5.164 s, 2192 MB heap size)
(2180 MB heap size)
(1874 MB heap size)
(2465 MB heap size)
(1996 MB heap size)
(1931 MB heap size)
(5.237 s, 1938 MB heap size)
(2252 MB heap size)
(5.079 s, 2247 MB heap size)
(2239 MB heap size)
(2057 MB heap size)
(1934 MB heap size)
(2525 MB heap size)
(1997 MB heap size)
(1990 MB heap size)
(2312 MB heap size)
(2307 MB heap size)
(2305 MB heap size)
(2116 MB heap size)
(2258 MB heap size)
(1995 MB heap size)
(2586 MB heap size)
(5.015 s, 2058 MB heap size)
(2050 MB heap size)
(2365 MB heap size)
(2373 MB heap size)
(2176 MB heap size)
(2390 MB heap size)
(2055 MB heap size)
(2337 MB heap size)
(2645 MB heap size)
(2118 MB heap size)
(2110 MB heap size)
(2433 MB heap size)
(2053 MB heap size)
(2237 MB heap size)
(2114 MB heap size)
(2450 MB heap size)
(2447 MB heap size)
(2178 MB heap size)
(2706 MB heap size)
(2180 MB heap size)
(2296 MB heap size)
(2175 MB heap size)
(2511 MB heap size)
(2507 MB heap size)
(2239 MB heap size)
(2767 MB heap size)
(2553 MB heap size)
(2241 MB heap size)
(2163 MB heap size)
(2235 MB heap size)
(2362 MB heap size)
(2567 MB heap size)
(5.929 s, 2598 MB heap size)
(2826 MB heap size)
(2613 MB heap size)
(2301 MB heap size)
(2293 MB heap size)
(2422 MB heap size)
(2301 MB heap size)
(2627 MB heap size)
(2284 MB heap size)
(2623 MB heap size)
(2887 MB heap size)
(2673 MB heap size)
(2286 MB heap size)
(2367 MB heap size)
(2369 MB heap size)
(2361 MB heap size)
(2687 MB heap size)
(2684 MB heap size)
(2419 MB heap size)
(2733 MB heap size)
(2963 MB heap size)
(2429 MB heap size)
(2427 MB heap size)
(2421 MB heap size)
(2747 MB heap size)
(2495 MB heap size)
(2489 MB heap size)
(2744 MB heap size)
(2793 MB heap size)
(2488 MB heap size)
(3007 MB heap size)
(2481 MB heap size)
(2807 MB heap size)
(2549 MB heap size)
(2804 MB heap size)
(2555 MB heap size)
(2854 MB heap size)
(2547 MB heap size)
(3067 MB heap size)
(2541 MB heap size)
(2867 MB heap size)
(2863 MB heap size)
(2615 MB heap size)
(2609 MB heap size)
(2607 MB heap size)
(2914 MB heap size)
(3127 MB heap size)
(2601 MB heap size)
(2927 MB heap size)
(2669 MB heap size)
(2667 MB heap size)
(2974 MB heap size)
(3187 MB heap size)
(2662 MB heap size)
(2681 MB heap size)
(2988 MB heap size)
(2729 MB heap size)
(2984 MB heap size)
(2727 MB heap size)
(2723 MB heap size)
(3247 MB heap size)
(2757 MB heap size)
(3048 MB heap size)
(2789 MB heap size)
(5.435 s, 3044 MB heap size)
(11.665 s, 2788 MB heap size)
(12.754 s, 2859 MB heap size)

I'll log out the code you posted to make sure it's being passed down.

Nov 12 '21 17:11 blimmer

Yep, @SimenB you were correct - the --no-sparkplug flag does appear to be making it down to the workers. Thanks for pointing me to that code.

@EternallLight and @rthreei - is there anything about your codebase that's notable to potentially cause this issue? The only thing I can think of on my end is that we make heavy use of async_hooks and have historically had issues upgrading in the 16.x series related to them.

I'm sure the jest team would love to have some kind of reproducibility to try to look more closely at this, but I'm struggling to develop a small reproducible case.

Nov 12 '21 17:11 blimmer

@blimmer nothing in particular is notable about our codebase. It's not a small codebase, but not very big either. Prior to 16.11, there were known memory leaks when running test suite (potentially around lodash); maybe made worse by 16.11.

Nov 12 '21 19:11 rthreei

If it's specifically in 16.11, you can probably try to build node yourself and bisect https://github.com/nodejs/node/compare/v16.10.0..v16.11.0. Figuring out which commit that introduced it might help understand when one (or more) of your code, Node and Jest does wrong 🙂

Nov 15 '21 12:11 SimenB

Looks like in new version of node, garbage collector will no longer clear memory until it reaches the limit.

RAM: 4GB

Node v14.17.3

node --expose-gc --trace_gc ./node_modules/.bin/jest app/javascript --forceExit --ci -w=2 --coverage --logHeapUsage > log/jest_memory_usage.log

Max memory usage:

[552:0x49fcc00]   116300 ms: Scavenge 700.3 (726.1) -> 692.0 (729.6) MB, 4.4 / 0.0 ms  (average mu = 0.983, current mu = 0.990) allocation failure

full log: https://gist.github.com/pustovalov/ad8bcd84b0b6bd6abec301982e03ed55

Node v16.13.0

node --expose-gc --trace_gc ./node_modules/.bin/jest app/javascript --forceExit --ci -w=2 --coverage --logHeapUsage > log/current-node-16.log

Max memory usage:

[10321:0x7fdd67100000]   865787 ms: Scavenge 2939.4 (3351.0) -> 2924.0 (3351.0) MB, 3.1 / 0.0 ms  (average mu = 0.346, current mu = 0.315) allocation failure

full log: https://gist.github.com/pustovalov/01cc1f484566f4b692fb405335b5f78d

Node v16.13.0 with `max-old-space-size`

based on how much memory the process was using when it used node v14, set a limit in 700mb

node --expose-gc --trace_gc --max-old-space-size=700 ./node_modules/.bin/jest app/javascript --forceExit --ci -w=2 --coverage --logHeapUsage > log/jest_memory_usage.log

Max memory usage:

[1040:0x66e7250]   193826 ms: Scavenge 603.6 (640.0) -> 592.4 (643.3) MB, 9.2 / 0.0 ms  (average mu = 0.823, current mu = 0.844) allocation failure

full log: https://gist.github.com/pustovalov/a86545aa38708d6dda479ee9a1bc4e1d

If you set a limit that will be equal to the maximum amount of RAM, then the process will crash because the cleaning will begin only when this limit is reached

  System:
    OS: macOS 11.6
    CPU: (8) x64 Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
  Binaries:
    Node: 16.13.0 - /usr/local/Cellar/node@16/16.13.0/bin/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 8.1.0 - /usr/local/Cellar/node@16/16.13.0/bin/npm
  npmPackages:
    jest: ^27.3.1 => 27.3.1

Nov 15 '21 15:11 pustovalov

If it's specifically in 16.11, you can probably try to build node yourself and bisect https://github.com/nodejs/node/compare/v16.10.0..v16.11.0. Figuring out which commit that introduced it might help understand when one (or more) of your code, Node and Jest does wrong 🙂

I've tried that. I was able to reproduce the issue on the last commit on v8 upgrade. It looks like the problem is related to v8 upgrade inside of node.

Nov 15 '21 17:11 luiz290788

Looks like I am hitting this as well. It happens only when doing coverage (as it eats more memory), and it ends up with hard failure. Locking node version to 16.10 helps, node 17 fails the same. Fails locally too.

Maybe one interesting note, it started to happen after I upgraded node-mongodb to v4, failed pipeline here.

Nov 16 '21 17:11 B4nan

Trying to find potentially related issues in NodeJS and this one jumped out at me: https://github.com/nodejs/node/issues/40014

And the associated V8 bug: https://bugs.chromium.org/p/v8/issues/detail?id=12198

Nov 16 '21 17:11 jgoz

I was trying to isolate the issue from our codebase and created this repo. It looks like that even for very simple test suites, the problem is there.

Nov 22 '21 15:11 luiz290788

We had consistent out of memory errors on node 16.13, reverting to 16.10 solved the problem for us.

Nov 23 '21 21:11 mousetree

Same here. On node 16.13.1, heap size goes up to 500MB while on node 16.10, it stays at 50MB with the following repro:

mkdir test
cd test
npm install jest
mkdir src
echo 'it("a", () => expect(0).toBeFalsy())' > src/a.test.js
for i in {1..100}; do cp src/a.test.js "src/a$i.test.js"; done
./node_modules/jest/bin/jest.js --runInBand --logHeapUsage

Edit: install Volta and do volta run --node 20.1.0 -- node ./node_modules/jest/bin/jest.js --runInBand --logHeapUsag to test a specific node version

Dec 02 '21 09:12 mb21

Running mb21's test on various Node versions confirms that everything after 16.10 (including 16.13.1 and 17.2.0) shows the same ever-growing heap, also running 3 times as slow.

We weren't able to get CI to finish running our tests under the latest Node 16, although somehow Node 17 was fine. 🤔

Dec 03 '21 22:12 LukeNotable

I've just tracked down this issue - for us it is a 2-5x performance regression (hopefully those key words help people find this). I've also tested the latest nodejs 16 & 17 versions without luck.

running a subset of tests, Memory in windows task manager on 16.10 reaches 900mb per worker. On 16.11.0 it reaches 1800mb per worker.

Dec 06 '21 10:12 lukeapage

Also hitting this after upgrading to Node 16.13. It's quite bad for us, as we cannot resolve some CVEs that have been fixed in 16.13 already.

I understand the underlying v8 issue is closed (wontfix), so I don't have much hope we'll have this fixed in a new node version any time soon.

So I am wondering:

Are there plans to solve/work around this issue somehow from the jest side?
Is anyone aware if this memory leak has implications beyond slowing down jest, i.e. should we hold back with updating node servers running in production?

Dec 09 '21 09:12 tzimmermann

Is anyone aware if this memory leak has implications beyond slowing down jest, i.e. should we hold back with updating node servers running in production?

I did some local profiling and didn't see any significant difference in memory consumption between 16.10 and 16.13 for our production app, but we aren't using vm.Script

Dec 09 '21 16:12 jgoz

Following a suggestion on the Node issue, I tried running with the Node option --no-compilation-cache. Suddenly my CI with 5000 tests works again, and furthermore it finishes without the ~1GB of memory leakage that I always saw under Node 14 (and of course without the 5GB+ that would leak under Node 16.13 until OOM). The downside is that it seems to take about 25–50% longer to finish.

Dec 11 '21 22:12 LukeNotable

We've been recently hit by this bug (or combination of them) in several services of our stack. It's been an inconvenience for use since once the heap size exceeded the 2GB available by GitHub Actions machines, our CI/CD process was failing 100% of the time.

I believe the "modules accumulating in the heap" bug has been present in Jest for some time, but that in combination with the changes in Node from v16.10 to v16.11 has made it much more severe. We're certain that bug exists in Jest, as we've managed to replicate it with a "dummy repo", as several other people have stated.

The fact that the ticket created in Node JS was closed as a WONTFIX is a bit worrying. I currently lack the knowledge to judge the discussions which happened there, but if that wasn't a bug at the end probably the solution falls on the Jest side.

After trying to extract as much information from the different GitHub issues that have been created in Jest, TS-Jest and Node, and trying several approaches, the only path for us was to downgrade back to Node v16.10.

These are some of the statistics we gathered in the process:

Node	`@jest/core`	isolatedModules	runInBand	Comp. cache	Initial H	Final H	H ratio	Time
17.2.0	27.3.1	true	true	true	133 MB	2236 MB	~33 MB	144 s
16.13.0	27.3.1	true	true	true	131 MB	2220 MB	~30 MB	130 s
16.13.0	27.3.1	true	true	false	126 MB	1712 MB	~25 MB	113 s
16.13.0	27.3.1	false	true	true	287 MB	2393 MB	~30 MB	140 s
16.10.0	27.3.1	true	true	true	123 MB	790 MB	~10 MB	77 s
16.10.0	27.3.1	true	true	false	118 MB	1676 MB	~25 MB	110 s
16.10.0	27.3.1	false	true	true	280 MB	952 MB	~10 MB	90 s

Some explanations about the columns:

Node: version of NodeJS
@jest/core: version of the Jest core
isolatedModules: the setting defined via jest.config.json
runInBand: specified by the --runInBand Jest flag
Comp. cache: specified by the NodeJS --no-compilation-cache flag
Initial H: heap size after running the first test
Final H: heap size after running the last test
H ratio: heap size increase after each test
Time: time the whole suite needed to run

The most relevant tickets related to this matter:

https://github.com/facebook/jest/issues/10550
https://github.com/facebook/jest/issues/11956
https://github.com/facebook/jest/issues/12142
https://github.com/facebook/jest/issues/7311
https://github.com/facebook/jest/issues/7874
https://github.com/kulshekhar/ts-jest/issues/1967
https://github.com/nodejs/node/issues/40014
https://bugs.chromium.org/p/v8/issues/detail?id=12198

We hope this issue is given the importance it deserves, as keeping the Node version pinned to v16.10 or relying on bigger CI machines to compensate this is a poor strategy for the future.

Dec 15 '21 15:12 amiedes

@amiedes it looks to me like the v8 bug was closed as won’t fix but the nodejs bug is still open.

I’m just a interested spectator and don’t have a 100% understanding but it seems to me like no one has indicated a belief this is a jest bug.

Dec 15 '21 16:12 lukeapage

@amiedes can you try to run jest in this way?

node --expose-gc --trace_gc --max-old-space-size=700 ./node_modules/.bin/jest app/javascript --runInBand --logHeapUsage > log/jest_memory_usage.log

Dec 15 '21 16:12 pustovalov

it looks to me like the v8 bug was closed as won’t fix but the nodejs bug is still open

@lukeapage thanks for pointing out the difference 👌 , I lost the details in the way and thought that was not being investigated anymore

can you try to run jest in this way?

@pustovalov these are the results from running:

node \
  --expose-gc \
  --trace_gc \
  --max-old-space-size=700 \
  ./node_modules/.bin/jest \
    --runInBand \
    --logHeapUsage \
    > jest_memory_usage.log

stdout:

PASS __tests__/routes/v3/sql/query/bigquery.test.ts (6.814 s, 221 MB heap size)
PASS __tests__/routes/v3/sql/query/jobs/bigquery.test.ts (282 MB heap size)
PASS __tests__/routes/v3/handlers/clients/bigquery.test.ts (327 MB heap size)
...more test lines...
PASS __tests__/routes/v3/maps/query/databricks.test.ts (624 MB heap size)
PASS __tests__/routes/v3/sql/query/snowflake.test.ts (646 MB heap size)

<--- Last few GCs --->

[30209:0x7f96af900000]    45289 ms: Mark-sweep 665.4 (738.9) -> 650.5 (739.4) MB, 468.1 / 0.4 ms  (average mu = 0.377, current mu = 0.374) allocation failure scavenge might not succeed
[30209:0x7f96af900000]    46075 ms: Mark-sweep 670.6 (741.3) -> 656.8 (744.0) MB, 600.1 / 0.2 ms  (average mu = 0.306, current mu = 0.237) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x101600515 node::Abort() (.cold.1) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 2: 0x100301989 node::Abort() [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 3: 0x100301aff node::OnFatalError(char const*, char const*) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 4: 0x1004812c7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 5: 0x100481263 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 6: 0x100622975 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 7: 0x1006269bd v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 8: 0x10062329d v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
 9: 0x1006207bd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
10: 0x10062daf0 v8::internal::Heap::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
11: 0x10062db71 v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
12: 0x1005f4c1d v8::internal::FactoryBase<v8::internal::Factory>::NewRawOneByteString(int, v8::internal::AllocationType) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
13: 0x1008cf8c1 v8::internal::String::SlowFlatten(v8::internal::Isolate*, v8::internal::Handle<v8::internal::ConsString>, v8::internal::AllocationType) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
14: 0x100765374 v8::internal::CompilationCacheTable::LookupScript(v8::internal::Handle<v8::internal::CompilationCacheTable>, v8::internal::Handle<v8::internal::String>, v8::internal::LanguageMode, v8::internal::Isolate*) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
15: 0x10051a42f v8::internal::CompilationCacheScript::Lookup(v8::internal::Handle<v8::internal::String>, v8::internal::ScriptDetails const&, v8::internal::LanguageMode) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
16: 0x100523cda v8::internal::Compiler::GetSharedFunctionInfoForScript(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>, v8::internal::ScriptDetails const&, v8::Extension*, v8::internal::ScriptData*, v8::ScriptCompiler::CompileOptions, v8::ScriptCompiler::NoCacheReason, v8::internal::NativesFlag) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
17: 0x10048b5ab v8::ScriptCompiler::CompileUnboundInternal(v8::Isolate*, v8::ScriptCompiler::Source*, v8::ScriptCompiler::CompileOptions, v8::ScriptCompiler::NoCacheReason) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
18: 0x1002f4b45 node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
19: 0x1004e9dd9 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
20: 0x1004e95ad v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<true>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
21: 0x1004e8ffb v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
22: 0x100d59eb9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
23: 0x100cea072 Builtins_JSBuiltinsConstructStub [/Users/amiedes/.nvm/versions/node/v16.13.0/bin/node]
zsh: abort      node --expose-gc --trace_gc --max-old-space-size=700 ./node_modules/.bin/jest

Memory usage log:

jest_memory_usage.log

Thanks you both a lot for taking a look into it 🙇

EDIT: I ran this with Node 16.13.0

Dec 15 '21 16:12 amiedes

@pustovalov thanks for taking a look! Just to make sure, you've seen my minimal repro above?

Dec 15 '21 16:12 mb21

@mb21 just looked, my current workaround for Node > 16.10 it's to tweak max-old-space-size, with your reproduce:

Node 16.13.1

./node_modules/jest/bin/jest.js --runInBand --logHeapUsage
PASS  src/a72.test.js (253 MB heap size)

Test Suites: 101 passed, 101 total
Tests:       101 passed, 101 total
Snapshots:   0 total
Time:        9.174 s, estimated 12 s

https://gist.github.com/pustovalov/c98f3f521943aa082a2edbc70a343302#file-1-log

node --expose-gc --max-old-space-size=50 ./node_modules/jest/bin/jest.js --runInBand --logHeapUsage
 PASS  src/a75.test.js (25 MB heap size)

Test Suites: 101 passed, 101 total
Tests:       101 passed, 101 total
Snapshots:   0 total
Time:        14.831 s

https://gist.github.com/pustovalov/c98f3f521943aa082a2edbc70a343302#file-2-log

In my case https://github.com/facebook/jest/issues/11956#issuecomment-969041221, limit in 700mb works fine for:

Test Suites: 150 passed, 150 total
Tests:       2159 passed, 2159 total
Snapshots:   259 passed, 259 total
Time:        332.784 s

vCPUs 2
RAM 4GB

Also the execution time increased

Node v14.17.3

Time: 240.665 s
Time: 314.481 s
Time: 270.317 s
Time: 240.255 s
Time: 335.224 s
Time: 239.925 s
Time: 260.644 s
Time: 334.419 s

Node v16.13.0

Time: 321.699 s
Time: 318.885 s
Time: 355.484 s
Time: 318.719 s
Time: 406.289 s

Dec 15 '21 18:12 pustovalov

For those interested, I'm inviting you to join us in a live discussion about this problem. We'll be online over Zoom for two hours on Friday at 8am ET to discuss how we might troubleshoot/fix this regression. Details below.

Calendar invite

https://calendar.google.com/event?action=TEMPLATE&tmeid=MWw1YWpoZGRpZXBsaGF2aWs0MGhndnA3MzkgcnNoYXJtYUBicml6YS5jb20&tmsrc=rsharma%40briza.com

Zoom link

Topic: Node.js 16.11.0+ / Jest - Memory consumption Time: Dec 17, 2021 08:00 AM Eastern Time (US and Canada)

Join Zoom Meeting https://briza.zoom.us/j/86836754356?pwd=SVI2dUxIVUlScGlRdEpFYXJtU3NCZz09

Meeting ID: 868 3675 4356 Passcode: 580759 One tap mobile +13126266799,,86836754356#,,,,*580759# US (Chicago) +13462487799,,86836754356#,,,,*580759# US (Houston)

Dial by your location +1 312 626 6799 US (Chicago) +1 346 248 7799 US (Houston) +1 646 558 8656 US (New York) +1 669 900 6833 US (San Jose) +1 253 215 8782 US (Tacoma) +1 301 715 8592 US (Washington DC) Meeting ID: 868 3675 4356 Passcode: 580759 Find your local number: https://briza.zoom.us/u/kdRQiLHyl

Dec 16 '21 01:12 rthreei

@rthreei Thanks for looking into this! 👍🏼

Any updates from the Zoom meeting to share?

Dec 20 '21 09:12 PAkerstrand

@PAkerstrand some notes:

The performance regression is likely contained to vm.Script and so running Node 16.11+ in production should not experience the performance regression.
The memory leak appears to be in the compilation cache.
Unless Jest is using vm.Script incorrectly, then the fix is unlikely to be in Jest space.
Speculation/hypothesis: The performance regression is likely in Node or v8.

What to do? Some options:

Upgrading and staying at Node 16.10 is not a viable option, since we won't be in a position to apply CVE patches.
Attempt to debug Node and/or v8. Possible, but hard.
Replace Jest with a testing framework that doesn't use vm.Script.

Dec 21 '21 00:12 rthreei

What worked for me was combining the 2 node flags: --no-compilation-cache and --expose-gc. Using this example above I've managed to run node --no-compilation-cache --expose-gc ./node_modules/jest/bin/jest.js --runInBand --logHeapUsage and got heap sizes between 22MB and 25MB. Also using these flags the max heap size in my codebase decreased from ~2000 MB to ~400MB . Tested on node v16.11.0. Hope it helps !

Dec 28 '21 09:12 oskar-kupski-elpassion

What worked for me was combining the 2 node flags: --no-compilation-cache and --expose-gc. Using this example above I've managed to run node --no-compilation-cache --expose-gc ./node_modules/jest/bin/jest.js --runInBand --logHeapUsage and got heap sizes between 22MB and 25MB. Also using these flags the max heap size in my codebase decreased from ~2000 MB to ~400MB . Tested on node v16.11.0. Hope it helps !

Did this increase execution time @oskar-kupski-elpassion ?

Dec 28 '21 14:12 rthreei

jest jest copied to clipboard

[Bug]: Memory consumption issues on Node JS 16.11.0+

Version

Steps to reproduce

Expected behavior

Actual behavior

Additional context

Environment

Node 16.10.0 (baseline)

Node 16.11.0 (problem introduced)

Node v14.17.3

Node v16.13.0

Node v16.13.0 with max-old-space-size

Calendar invite

Zoom link

jest
jest copied to clipboard

Node v16.13.0 with `max-old-space-size`