jest icon indicating copy to clipboard operation
jest copied to clipboard

[Bug]: Jest 30 seems to be 1.5x slower than v29

Open bacarybruno opened this issue 5 months ago • 25 comments

Version

30.0.3

Steps to reproduce

  • Clone the facebook/react repo
  • Run yarn install
  • Run the tests with yarn test
  • Bump jest to v30.0.3
  • Run the tests again

Expected behavior

According to the blogpost https://jestjs.io/blog/2025/06/04/jest-30, "Jest 30 is noticeably faster" than the previous versions.

Actual behavior

I attempted an upgrade on two private repositories and noticed that on both of them, jest 30 was slower.

Jest 29: Time: 57.615 s Jest 30: Time: 90.925 s

Additional context

Image

Environment

System:
    OS: macOS 15.5
    CPU: (12) arm64 Apple M4 Pro
  Binaries:
    Node: 20.19.0 - ~/.nvm/versions/node/v20.19.0/bin/node
    Yarn: 1.22.22 - ~/.nvm/versions/node/v20.19.0/bin/yarn
    npm: 10.8.2 - ~/.nvm/versions/node/v20.19.0/bin/npm
  npmPackages:
    jest: ^30.0.3 => 30.0.3

bacarybruno avatar Jul 02 '25 15:07 bacarybruno

My team is noticing the same thing

Jest 29: Time: ~130s Jest 30: Time: ~170s

Codexphere92 avatar Jul 05 '25 17:07 Codexphere92

Us too

Jest 29: ~97s Jest 30: ~133s

tavvy avatar Jul 07 '25 11:07 tavvy

Me too.

Jest 29: ~72s Jest 30: ~125s

Ubuntu 24.04, 11th Gen Intel® Core™ i7-1165G7 × 8, 32GB RAM

henriquekuwai avatar Jul 07 '25 13:07 henriquekuwai

I did some investigating and found that the performance regression was introduced between 30.0.0-beta.3 and 30.0.0-beta.4.

Based on the diff https://github.com/jestjs/jest/compare/v30.0.0-beta.3...v30.0.0-beta.4, it looks like this PR (feat(runtime): use compileFunction over new Script) could be the likely cause cc @SimenB (sorry for the wild ping 🙏).

I was also expecting some performance improvements from this PR, but it didn’t seem to have any noticeable impact on run durations 🤔 Is there anything to update on the config maybe to make it work?

bacarybruno avatar Jul 08 '25 06:07 bacarybruno

Can anyone upload/link to a repo for reproducing this other than React?

I can't even seem to upgrade Jest in React's repo. I'm encountering this issue and many other errors. The setup in this repo is very complicated.

eyalroth avatar Jul 12 '25 15:07 eyalroth

Sorry I struggle to find a big open-source repo with significant test duration. On https://github.com/Shopify/polaris for example I had to use the --runInBand option to see a difference:

v29.7.0 Image

v30.0.3 Image

Without runInBand the difference was not that big (jest 29 was 2-3s slower).

bacarybruno avatar Jul 16 '25 02:07 bacarybruno

Observing the same issue here: RAM usage increased, and tests take way longer Probably related to: https://github.com/nodejs/node/issues/35375

aclarembeau avatar Jul 17 '25 12:07 aclarembeau

On Shopify/polaris for example I had to use the --runInBand option to see a difference:

I'm assuming you were referring to the tests in polaris-react folder.

I got the same result with Node 20 (and running Jest 30 from my local clone of master, not the published version).

However, with Node 22, Jest 30 was faster (~25s) than both Jest versions in Node 20, while Jest 29 gotten worse (~35s).

What version of node are people here using in their projects that experience the slowdown?

eyalroth avatar Jul 17 '25 16:07 eyalroth

I'm assuming you were referring to the tests in polaris-react folder.

Yes 👍

What version of node

We are using Node 20 on our private repo

bacarybruno avatar Jul 17 '25 18:07 bacarybruno

We use node 22

aclarembeau avatar Jul 18 '25 06:07 aclarembeau

@eyalroth is there something else we can do to help? 🙏

bacarybruno avatar Jul 24 '25 07:07 bacarybruno

@eyalroth is there something else we can do to help? 🙏

@bacarybruno I contributed a few major features in Jest 30, so I was interested to see if they may be the cause of this issue.

Unfortunately, I couldn't find any relation between my contributions and the issue.

This release was very big with many different contributions.

eyalroth avatar Jul 24 '25 16:07 eyalroth

To contribute another data point, we can observe the slowdown in our private repo - we use --runInBand and the affected test suite is an end-to-end one with a latest NestJS + MikroORM + Postgres + supertest setup, so there's quite a bit of HTTP and setup/teardown code running (we use actual database and clear most of the data via ORM per each test).

We are running latest Node 22 and can reproduce the slowdown on a Pop!_OS 22.04 machine (AMD Ryzen 3900X), can definitely observe the slowdown in the ~33-50% range.

Xanewok avatar Jul 28 '25 20:07 Xanewok

We've just upgraded from Jest 29 to 30. This upgrade significantly increased the performance for our big library package and decreased it for two apps. Adding this to the jest.config.cjs at least returned the v29 run time:

const defaultTestConfig = {
  testEnvironmentOptions: {
    globalsCleanup: 'on'
  },
  // other configs
}

module.exports = defaultTestConfig;

Maybe this will help someone to upgrade without suffering from the performance penalties.

Based on: https://jestjs.io/blog/2025/06/04/jest-30#globals-cleanup-between-test-files

maksymz avatar Jul 30 '25 20:07 maksymz

A pretty massive change for us. This is on our backend tests, react tests are even worse. And this is only one package of tests, we have over 50 of these.

Image to Image

Heap usage also went from around 900MB per test to over 2500MB per test.

Node: v22.12.0 Debian bookworm 12.11 Container on an Apple M2 Max / 64GB of Ram

jasonmacdonald avatar Jul 31 '25 19:07 jasonmacdonald

Same here, i am observing slower time using node 20/ jest 30 on azure CI. I managed to get a bit closer to the Jest 29 time execution by forcing --runInBand and globalsCleanup: 'on' but it is still a bit slower.

boubou158 avatar Aug 06 '25 06:08 boubou158

Has anyone managed to get faster tests after switching to jest 30? Or what projects were the official announcement based on? Maybe we'll be able to figure out what conditions need to be met to make them faster. I the meantime we'll keep jest 29.

bacarybruno avatar Aug 16 '25 07:08 bacarybruno

Same issue for us.

Jest 29.5.0 - Node cimg/node:20.14.0 - CI run

Image

Jest 30.0.5 - Node cimg/node:20.14.0 - CI run

Image

carta-quentin-crinon avatar Aug 21 '25 13:08 carta-quentin-crinon

Seeing this as well. An increase in ~27% from our Jest 29 average test execution.

cartond avatar Aug 25 '25 17:08 cartond

As another data point, in our React test suite, I am now seeing a ~15min duration for Jest 30 compared to ~12min with Jest 29. For context, I have not yet enabled any other Jest 30 features (i.e. globalCleanup).

giavinh79 avatar Sep 03 '25 21:09 giavinh79

Same issue for us in Jest for Angular. I was checking many options and this the one that impact most, particulary isolatedModules: true. This property means that Jest compiles each TypeScript file independently without performing a complete type check. Trade-off: Doesn't catch type errors during testing (your IDE and build process do that anyway).

I have to see if there are any other disadvantages, but it significantly reduces the time.

    transform: {
    '^.+\\.(ts|js|mjs|html|svg)$': [
      'jest-preset-angular',
      {
        tsconfig: '<rootDir>/tsconfig.spec.json',
        stringifyContentPathRegex: '\\.(html|svg)$',
        isolatedModules: true,
      },
    ],
  },

adrian-orquest avatar Oct 03 '25 07:10 adrian-orquest

Can anyone from the jest team provide more context on this or potential workarounds? It's pretty frustrating to see 33% perf gains announced and to spend lots of time working through breaking changes only to see this perf slowdown

jordan-cutler avatar Oct 18 '25 15:10 jordan-cutler

Also seeing 15-20% slowdown across small and large test suites in our codebases which has paused upgrading.

tylerkrupicka-stripe avatar Oct 29 '25 17:10 tylerkrupicka-stripe

In lack of an official response in almost 4 months, I'll share what we did to get back down to our original times finally. Hopefully nothing sends you down the wrong rabbit hole, but if you're desperate.... It was a combination of things in no particular order:

  1. Extensive resource testing. We ran up to 300 iterations on ~10 combinations playing with the max workers and RAM levers 1.a. --maxWorkers=xx [docs] and --max-old-space-size=xxxxxx or percentage on node [docs]. Note, jest takes one thread to orchestrate by default. 1.b. using Honeycomb made this very easy to visualize and consolidate the metrics. highly recommend using something like that 1.c. Everyone's CI is different so you will have to figure this out on your own. I'd first look at what the instance has for RAM and vCPU, and ssh in at idle to see what is actually available.
  2. Updating all our custom SWC versions to be more up to date. I'm not totally sure about this one but probably some Rust magic
  3. Remove --verbose by default, and added support to enable it with env vars
  4. Utilize --workerIdleMemoryLimit=xxxxMB [docs]
  5. Using cacheDirectory: <some-path> [docs] seemed benefit CI on intra-run caching 5.a. this was probably the biggest helper because our test suite is SO large 5.b. you can validate this helping you by doing some runs with cache disabled entirely --no-cache

I was going to try and really help by using the react repo to apply/show these in action, but when I yarn test I get ~225 test failures, sorry.

cartond avatar Oct 31 '25 15:10 cartond

TL;DR:

  1. Apparently no issues on 30.0.0-alpha.6.
  2. We first encountered the issue on 30.0.5 (not sure about previous versions).
  3. Issue becomes relevant on CI, almost certainly because of the number of involved workers (1 against 15 in below benchmarks).
  4. 30.0.5 introduces a massive slowdown (on CI), of around 53.5%.
  5. 30.2.0 goes out of memory (on CI).
  6. There is no slowdown with high number of involved workers and plenty of available memory.

Benchmarks:

  • PLATFORM: ubuntu (GitHub Actions)
  • CPU: 2 cores
  • MEMORY: 7 GB
  • NODE: 22.6.0
  • JEST: 30.0.0-alpha.6

jest --ci --watch=false --coverage=true --bail [--max-workers=1 (default/auto)]

Test Suites: 470 passed, 470 total
Tests:       3071 passed, 3071 total
Snapshots:   271 passed, 271 total
Time:        360.007 s
Ran all test suites in 4 projects.

  • PLATFORM: ubuntu (GitHub Actions)
  • CPU: 2 cores
  • MEMORY: 7 GB
  • NODE: 22.6.0
  • JEST: 30.0.5

jest --ci --watch=false --coverage=true --bail [--max-workers=1 (default/auto)]

RESULT: TIMEOUT AFTER 10 min (600s) (workflow timeout)

-> 352 out of 470 test suited completed (~75%) -> ~53.5% slower


  • PLATFORM: ubuntu (GitHub Actions)
  • CPU: 2 cores
  • MEMORY: 7 GB
  • NODE: 22.6.0
  • JEST: 30.2.0

jest --ci --watch=false --coverage=true --bail [--max-workers=1 (default/auto)]

RESULT: OUT OF MEMORY

-> 131 out of 470 test suited completed (~28%) -> ~3.8% slower (not reliable because of the number of completed tests)

<--- Last few GCs --->

[2261:0x3cd46000]   382198 ms: Scavenge (interleaved) 1901.1 (2078.7) -> 1890.4 (2080.7) MB, pooled: 5 MB, 31.43 / 0.00 ms  (average mu = 0.307, current mu = 0.301) allocation failure;
[2261:0x3cd46000]   384204 ms: Mark-Compact 1905.3 (2080.7) -> 1886.7 (2076.5) MB, pooled: 9 MB, 1987.72 / 0.00 ms  (average mu = 0.241, current mu = 0.164) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xe2f0f2 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]
 2: 0x1240cf0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]
 3: 0x1240fc7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]
 4: 0x1470715  [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]
 5: 0x1470743  [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]
 6: 0x14897fa  [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]
 7: 0x148c9c8  [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]
 8: 0x1e25421  [/opt/hostedtoolcache/node/22.6.0/x64/bin/node]

  • PLATFORM: macOS
  • CPU: Apple M3 Max
  • MEMORY: 48 GB
  • NODE: 22.6.0
  • JEST: 30.0.0-alpha.6

jest --ci --watch=false --coverage=true --bail -u [--max-workers=15 (default/auto)]

Test Suites: 470 passed, 470 total
Tests:       3071 passed, 3071 total
Snapshots:   271 passed, 271 total
Time:        21.854 s
Ran all test suites in 4 projects.

  • PLATFORM: macOS
  • CPU: Apple M3 Max
  • MEMORY: 48 GB
  • NODE: 22.6.0
  • JEST: 30.2.0

jest --ci --watch=false --coverage=true --bail -u [--max-workers=15 (default/auto)]

Test Suites: 470 passed, 470 total
Tests:       3073 passed, 3073 total
Snapshots:   271 passed, 271 total
Time:        21.944 s
Ran all test suites in 4 projects.

  • PLATFORM: macOS
  • CPU: Apple M3 Max
  • MEMORY: 48 GB
  • NODE: 22.6.0
  • JEST: 30.0.0-alpha.6

jest --ci --watch=false --coverage=true --bail -u --max-workers=1

Test Suites: 470 passed, 470 total
Tests:       3071 passed, 3071 total
Snapshots:   271 passed, 271 total
Time:        100.163 s, estimated 276 s
Ran all test suites in 4 projects.

  • PLATFORM: macOS
  • CPU: Apple M3 Max
  • MEMORY: 48 GB
  • NODE: 22.6.0
  • JEST: 30.2.0

jest --ci --watch=false --coverage=true --bail -u --max-workers=1

Test Suites: 470 passed, 470 total
Tests:       3073 passed, 3073 total
Snapshots:   271 passed, 271 total
Time:        138.348 s
Ran all test suites in 4 projects.

This is unfortunate because Jest 30 was advertised to have "massive performance boosts" and to be "Faster, Leaner, Better".

ernestostifano avatar Nov 06 '25 12:11 ernestostifano