next.js icon indicating copy to clipboard operation
next.js copied to clipboard

Nextjs with jest example will not work if there is an esm dependency in node_modules

Open wmira opened this issue 3 years ago • 28 comments

Verify canary release

  • [X] I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
  Platform: linux
  Arch: x64
  Version: #1 SMP Fri Apr 2 22:23:49 UTC 2021
Binaries:
  Node: 16.16.0
  npm: 8.11.0
  Yarn: 1.22.19
  pnpm: N/A
Relevant packages:
  next: 12.2.5
  eslint-config-next: 12.2.5
  react: 18.2.0
  react-dom: 18.2.0

Which example does this report relate to?

https://nextjs.org/docs/testing

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

If you have a project with a dependency that is published as an es modules in node_modules, running jest will not work..even though nextjs works with it e.g. running the project normally with npm run dev.

Test does not run even if ignoring the package. Please check this repo that reproduce the problem:

https://github.com/wmira/banana/

if you run npm run dev here..works fine.. npm run test here does not and this follow the jest example.

Expected Behavior

running test via jest works as per the docs

To Reproduce

repo is below: https://github.com/wmira/banana/

  1. npm run dev -- nextjs works fine with an es module dependency under node_modules
  2. npm run test -- throws error

wmira avatar Sep 02 '22 16:09 wmira

I ran into this issue once I installed preact in to my project.

bruceharrison1984 avatar Sep 04 '22 19:09 bruceharrison1984

Same issue for me. I've fixed with:

module.exports = async () => ({
  ...(await createJestConfig(customJestConfig)()),
  transformIgnorePatterns: ["node_modules/(?!(package1|package2)/)", "^.+\\.module\\.(css|sass|scss)$"],
});

In my case packages are: swiper|ssr-window|dom7 Any suggestion?

dariolongo avatar Sep 15 '22 10:09 dariolongo

I had the same.

The cause of this issue is that I cannot include any module under node_modules in Jest's trasform target.

Specifically,

  • /node_modules/ is defined in transformIgnorePatterns by next/jest.
  • customJestConfig cannot override it and is limited to adding it

This is the relevant part of next/jest. https://github.com/vercel/next.js/blob/v12.3.0/packages/next/build/jest/jest.ts#L136

The solution offered by @dariolongo is adequate. However, instead of completely overriding it, a diversion of the standard ignorePatterns might be safer.

const esModules = ['package1', 'package2']
const customConfig = {
  // Dependency packages must also be included in this list
  transformIgnorePatterns: [`/node_modules/(?!(${esModules.join('|')})/)`],
  // And other custom config...
}

module.exports = async () => {
  const jestConfig = await _createJestConfig(customConfig)()
  return {
    ...jestConfig,
    transformIgnorePatterns: jestConfig.transformIgnorePatterns.filter(
      (ptn) => ptn !== '/node_modules/'
    ), // ['^.+\\.module\\.(css|sass|scss)$', '/node_modules/(?!(package1|package2)/']
  }
}

I would like to suggest that the documentation be supplemented with "config cannot be completely overwritten".

nora avatar Sep 16 '22 08:09 nora

I've been trying to look into the next.js configuration, and from what I can see, node_modules isn't excluded in the standard next.js config when using esm and swc (both defaults as of Next 12), so I'm suspecting that this might just be a left over from a pre-ESM time?

ThisIsMissEm avatar Oct 11 '22 19:10 ThisIsMissEm

Solution from @nora worked perfectly. I believe this is a bug, but at the very least the docs should be updated to show how transformIgnorePatterns can be configured. What's currently implied in the docs is incorrect.

BoilingSoup avatar Jan 15 '23 02:01 BoilingSoup

The solution offered by @dariolongo is adequate. However, instead of completely overriding it, a diversion of the standard ignorePatterns might be safer.

const esModules = ['package1', 'package2']
const customConfig = {
  // Dependency packages must also be included in this list
  transformIgnorePatterns: [`/node_modules/(?!(${esModules.join('|')})/)`],
  // And other custom config...
}

module.exports = async () => {
  const jestConfig = await _createJestConfig(customConfig)()
  return {
    ...jestConfig,
    transformIgnorePatterns: jestConfig.transformIgnorePatterns.filter(
      (ptn) => ptn !== '/node_modules/'
    ), // ['^.+\\.module\\.(css|sass|scss)$', '/node_modules/(?!(package1|package2)/']
  }
}

I would like to suggest that the documentation be supplemented with "config cannot be completely overwritten".

When using Next.js 13.1 transpilePackages, this solution seems not working by another regex next/jest added.

But now just put all esModules into transpilePackages in next.config.js works perfectly for me!

module.exports = {
  transpilePackages: ['package1', 'package2'],
}

oosawy avatar Feb 09 '23 05:02 oosawy

fyi i fixed this issue by removing the default paths in tsconfig "paths": { "": ["./"] }

sshah98 avatar Apr 07 '23 20:04 sshah98

The solution offered by @dariolongo is adequate. However, instead of completely overriding it, a diversion of the standard ignorePatterns might be safer.

const esModules = ['package1', 'package2']
const customConfig = {
  // Dependency packages must also be included in this list
  transformIgnorePatterns: [`/node_modules/(?!(${esModules.join('|')})/)`],
  // And other custom config...
}

module.exports = async () => {
  const jestConfig = await _createJestConfig(customConfig)()
  return {
    ...jestConfig,
    transformIgnorePatterns: jestConfig.transformIgnorePatterns.filter(
      (ptn) => ptn !== '/node_modules/'
    ), // ['^.+\\.module\\.(css|sass|scss)$', '/node_modules/(?!(package1|package2)/']
  }
}

I would like to suggest that the documentation be supplemented with "config cannot be completely overwritten".

When using Next.js 13.1 transpilePackages, this solution seems not working by another regex next/jest added.

But now just put all esModules into transpilePackages in next.config.js works perfectly for me!

module.exports = {
  transpilePackages: ['package1', 'package2'],
}

As mentioned by @oosawy, with the use of Next.js 13.1's transpilePackages, the regex has changed and will generate these transformIgnorePatterns:

module.exports = {
  transpilePackages: ['package1', 'package2'],
}
[
  '/node_modules/(?!.pnpm)(?!(package1|package2)/)',
  '/node_modules/.pnpm/(?!(package1|package2)@)',
  '^.+\\.module\\.(css|sass|scss)$'
]

In this case, we don't need to add transformIgnorePatterns in the custom configuration, but, I need to remove the first element from the generated transformIgnorePatterns array to make it work.

module.exports = async () => {
  const jestConfig = await _createJestConfig(customConfig)()
  return {
    ...jestConfig,
    transformIgnorePatterns: jestConfig.transformIgnorePatterns.filter(
      (ptn) => ptn !== '/node_modules/(?!.pnpm)(?!(package1|package2)/)'
    ),
  }
}

tachun avatar Apr 12 '23 17:04 tachun

This workaround seems to be broken with the release of 13.4. Has anyone else had luck upgrading to that release?

drewloomer avatar May 09 '23 16:05 drewloomer

Also encountering this issue. Wondering if next jest config could take into account type: "module" of the node_modules dependencies and auto add them into transformIgorePatterns?

jonioni avatar May 10 '23 06:05 jonioni

I am experiencing the same issue with an ESM node_module causing Jest errors in v.13.4.1 and ended up downgrading to v.13.2.3. Tried different combinations of transpilePackages in next.config.mjs and transformIgnorePatterns in jest.config.mjs but finally gave up and downgraded the version.

LarsEjaas avatar May 11 '23 21:05 LarsEjaas

I just ran into this bug for the query-string module. I found that adding query-string and all of it's dependencies to transpilePackages worked.

  transpilePackages: [
    "query-string",
    "decode-uri-component",
    "filter-obj",
    "split-on-first",
  ],

Then I didn't need to modify transformIgnorePatterns at all. The default:

["/node_modules/(?!.pnpm)(?!(query-string|decode-uri-component|filter-obj|split-on-first)/)","/node_modules/.pnpm/(?!(query-string|decode-uri-component|filter-obj|split-on-first)@)","^.+\\.module\\.(css|sass|scss)$"]

was correct. This might just be the way it has to be if using npm. Switching to pnpm might allow you to only include the top-level package, query-string, in my case, to transpilePackages because of the different way that pnpm bundles packages. It looks like that's what the second generated pattern is for, but I haven't tested that.

brett-minca avatar Jun 01 '23 18:06 brett-minca

Has this been resolved? I am still unable to use Jest with nextjs in 13.4 due to esm issues in node_modules.

AlfredMadere avatar Aug 08 '23 20:08 AlfredMadere

Dittoing @AlfredMadere, I'm also still unable to use Jest with Next.JS in 13.5.4 due to esm issues.

paulhchoi avatar Oct 06 '23 04:10 paulhchoi

i tried everything in this issue and i`m stuck forever here

leandroruel avatar Nov 01 '23 15:11 leandroruel

@leandroruel try converting your ESM dependencies to CJS for Jest:

npx esbuild [YOUR_DEP] --bundle --platform=node --outfile=vendor/[YOUR_DEP].js

Then update your jest.config.js:

moduleNameMapper: {
    '^[YOUR_DEP]$': '<rootDir>/vendor/[YOUR_DEP]',
    ...
}

twobit avatar Nov 01 '23 17:11 twobit

~~I think I'll stick with babel until this one is resolved - https://nextjs.org/docs/pages/building-your-application/optimizing/testing#setting-up-jest-with-babel~~ The babel example was removed from the documentation.

atstoyanov avatar Nov 06 '23 08:11 atstoyanov

This works for me! using next.js 13.5.6

module.exports = {
  transpilePackages: ['package1', 'package2'],
}

But the error a changed to:

@font-face {
    ^

    SyntaxError: Invalid or unexpected token

vicasas avatar Dec 04 '23 15:12 vicasas

The solution offered by @dariolongo is adequate. However, instead of completely overriding it, a diversion of the standard ignorePatterns might be safer.

const esModules = ['package1', 'package2']
const customConfig = {
  // Dependency packages must also be included in this list
  transformIgnorePatterns: [`/node_modules/(?!(${esModules.join('|')})/)`],
  // And other custom config...
}

module.exports = async () => {
  const jestConfig = await _createJestConfig(customConfig)()
  return {
    ...jestConfig,
    transformIgnorePatterns: jestConfig.transformIgnorePatterns.filter(
      (ptn) => ptn !== '/node_modules/'
    ), // ['^.+\\.module\\.(css|sass|scss)$', '/node_modules/(?!(package1|package2)/']
  }
}

I would like to suggest that the documentation be supplemented with "config cannot be completely overwritten".

When using Next.js 13.1 transpilePackages, this solution seems not working by another regex next/jest added.

But now just put all esModules into transpilePackages in next.config.js works perfectly for me!

module.exports = {
  transpilePackages: ['package1', 'package2'],
}

Above suggestion worked for me with Nextjs 14.0.3.

ankita-khandelwal avatar Dec 11 '23 03:12 ankita-khandelwal

I am on NextJS 14.0.3 and the transpilePackages solution did not work for me for package next-image-export-optimizer. ~~I got it working by switching back to babel as indicated by @atstoyanov.~~ nvm, it didn't work with babel either.

esantosrvolvo avatar Dec 12 '23 09:12 esantosrvolvo

It work in 14.0.3 using transpilePackages and without the need of overriding the transformIgnorePatterns. Here is my configuration.

// next.config.js
{
  transpilePackages: [
    'axios',
    'react-dnd-html5-backend',
    'supports-webp',
    'ramda',
    'react-firebase-hooks/auth',
  ]
}

// jest.config.ts
import type { Config } from 'jest';
import nextJest from 'next/jest.js';

const createJestConfig = nextJest({
  // Provide the path to your Next.js app to load next.config.js and .env files in your test environment
  dir: './',
});

// Add any custom config to be passed to Jest
const config: Config = {
  coverageProvider: 'v8',
  testEnvironment: 'jsdom',
  setupFilesAfterEnv: ['<rootDir>/jest.setup.js'],
  moduleNameMapper: {
    'react-markdown': '<rootDir>/node_modules/react-markdown/react-markdown.min.js',
    '^@/(.*)$': '<rootDir>/$1',
  },
  testPathIgnorePatterns: ['/tests/'],
  resetMocks: false,
  clearMocks: true,
};

export default createJestConfig(config);

and here is the produced config:

{
  coverageProvider: 'v8',
  testEnvironment: 'jsdom',
  setupFilesAfterEnv: [ '<rootDir>/jest.setup.js' ],
  moduleNameMapper: {
    '^.+\\.module\\.(css|sass|scss)$': '/Users/user/src/demo-app/node_modules/next/dist/build/jest/object-proxy.js',
    '^.+\\.(css|sass|scss)$': '/Users/user/src/demo-app/node_modules/next/dist/build/jest/__mocks__/styleMock.js',
    '^.+\\.(png|jpg|jpeg|gif|webp|avif|ico|bmp)$': '/Users/user/src/demo-app/node_modules/next/dist/build/jest/__mocks__/fileMock.js',
    '^.+\\.(svg)$': '/Users/user/src/demo-app/node_modules/next/dist/build/jest/__mocks__/fileMock.js',
    '@next/font/(.*)': '/Users/user/src/demo-app/node_modules/next/dist/build/jest/__mocks__/nextFontMock.js',
    'next/font/(.*)': '/Users/user/src/demo-app/node_modules/next/dist/build/jest/__mocks__/nextFontMock.js',
    'server-only': '/Users/user/src/demo-app/node_modules/next/dist/build/jest/__mocks__/empty.js',
    'react-markdown': '<rootDir>/node_modules/react-markdown/react-markdown.min.js',
    '^@/(.*)$': '<rootDir>/$1'
  },
  testPathIgnorePatterns: [ '/node_modules/', '/.next/', '/tests/' ],
  resetMocks: false,
  clearMocks: true,
  transform: {
    '^.+\\.(js|jsx|ts|tsx|mjs)$': [
      '/Users/user/src/demo-app/node_modules/next/dist/build/swc/jest-transformer.js',
      [Object]
    ]
  },
  transformIgnorePatterns: [
    '/node_modules/(?!.pnpm)(?!(axios|react-dnd-html5-backend|supports-webp|ramda|react-firebase-hooks/auth)/)',
    '/node_modules/.pnpm/(?!(axios|react-dnd-html5-backend|supports-webp|ramda|react-firebase-hooks\\+auth)@)',
    '^.+\\.module\\.(css|sass|scss)$'
  ],
  watchPathIgnorePatterns: [ '/.next/' ]
}

atstoyanov avatar Jan 11 '24 12:01 atstoyanov

I have similar issue. I just upgraded to nextjs 14 from nextjs 12. And i get the following error:

Cannot find module '../components/base/animation/animations' from 'src/lib/utils.js'

Inside my jsconfig.js i have "css/animations": ["./src/components/base/animation/animations.module.css"], and I use it like import animations from "css/animations";

It works fine when i run the app, but jest throws errors when i run the test.

YousefMohsen avatar Feb 20 '24 12:02 YousefMohsen

I'm getting this issue with Next 14.1.0, even when using create-next-app. It looks like ESLINT and JEST/Cypress are not working together.

It is possible to quickly reproduce it following the steps below:

Run npx create-next-app@latest (Choose the option to include Eslint on the config)

Then, follow the step to add config Jest https://nextjs.org/docs/app/building-your-application/testing/jest

Then, try to run the yarn test command. The following bug will be shown image

I have tried to use transpilePackages as mentioned above but without success. Any idea?

RobsonAraujo avatar Feb 24 '24 16:02 RobsonAraujo

if it helps someone here's my config file:

const nextJest = require('next/jest')
const swiperMocks = require('./src/__mocks__/misc/swiper.ts')

const createJestConfig = nextJest({
  // Path to Next.js app to load next.config.js
  dir: './',
})

/** @type {import('@jest/types').Config.InitialOptions} */
const customJestConfig = {
  testEnvironment: 'jest-environment-jsdom',
  setupFilesAfterEnv: ['<rootDir>/jest.setup.ts'],
  maxWorkers: 4,
  /**
   * Custom config goes here, I am not adding it to keep this example simple.
   * See next/jest examples for more information.
   */
  moduleNameMapper: {
    '^@/components/(.*)$': '<rootDir>/components/$1',
    '^components/(.*)$': '<rootDir>/src/components/$1',
    '^helpers/(.*)$': '<rootDir>/src/helpers/$1',
    '^generated/(.*)$': '<rootDir>/src/generated/$1',
    '^mixins/(.*)$': '<rootDir>/src/mixins/$1',
    '^hooks/(.*)$': '<rootDir>/src/hooks/$1',
    '^mocks/(.*)$': '<rootDir>/src/__mocks__/$1',
    ...swiperMocks.moduleNameMapper,
  },
  transform: {
    ...swiperMocks.transform,
  },
  modulePaths: ['<rootDir>/src/'],
  collectCoverage: true,
  collectCoverageFrom: [
    '<rootDir>/src/components/**/*.tsx', 
    '!<rootDir>/src/pages/**',
    '!<rootDir>/src/components/**/stories.tsx', 
    '!<rootDir>/src/components/**/index.ts',
  ],
  coveragePathIgnorePatterns: ['.*__snapshots__/.*'],
}

const esModules = [
  'ccount',
  'react-markdown',
  'vfile',
  'unist-.+',
  'unified',
  'bail',
  'is-plain-obj',
  'trough',
  'remark',
  'remark-.+',
  'mdast-util-.+',
  'micromark',
  'parse-entities',
  'character-entities',
  'property-information',
  'comma-separated-tokens',
  'hast-util-whitespace',
  'hast-util-to-html',
  'hast-util-sanitize',
  'html-void-elements',
  'space-separated-tokens',
  'decode-named-character-reference',
  'zwitch',
  'longest-streak',
  'stringify-entities',
  'trim-lines',
  'swiper',
  'swiper/react',
  'ssr-window',
  'dom7',
].join('|')

module.exports = async () => ({
  /**
   * Using ...(await createJestConfig(customJestConfig)()) to override transformIgnorePatterns
   * provided byt next/jest.
   *
   * @link https://github.com/vercel/next.js/issues/36077#issuecomment-1096635363
   */
  ...(await createJestConfig(customJestConfig)()),
  /**
   * Swiper uses ECMAScript Modules (ESM) and Jest provides some experimental support for it
   * but "node_modules" are not transpiled by next/jest yet.
   *
   * @link https://github.com/vercel/next.js/issues/36077#issuecomment-1096698456
   * @link https://jestjs.io/docs/ecmascript-modules
   */
  transformIgnorePatterns: [`<rootDir>/node_modules/(?!${esModules})/`],
})

leandroruel avatar Mar 05 '24 20:03 leandroruel

I'm getting this issue with Next 14.1.0, even when using create-next-app. It looks like ESLINT and JEST/Cypress are not working together.

It is possible to quickly reproduce it following the steps below:

Run npx create-next-app@latest (Choose the option to include Eslint on the config)

Then, follow the step to add config Jest https://nextjs.org/docs/app/building-your-application/testing/jest

Then, try to run the yarn test command. The following bug will be shown image

I have tried to use transpilePackages as mentioned above but without success. Any idea?

I was able to fix that by adding

"resolutions": {
    "string-width": "4.2.3",
    "wrap-ansi": "7.0.0"
  }

in my package.json. Not quite sure if it's the best solution but it fixed the problem :)

Make sure you do rm -rf node_modules and then re-run yarn.

AtanasChachev avatar May 09 '24 08:05 AtanasChachev

@AtanasChachev That works; thanks for sharing this alternative solution 💯

RobsonAraujo avatar May 28 '24 01:05 RobsonAraujo

@leandroruel thank you for your config but this still does not work for me. I've no idea but it seems like the transformIgnorePatterns is still ignored and you can't mock any module that imports ESM.

Here are my deps:

  "dependencies": {
    "@auth/supabase-adapter": "^1.1.0",
    "@supabase/ssr": "^0.4.0",
    "@supabase/supabase-js": "^2.44.0",
    "jose": "^5.3.0",
    "jsonwebtoken": "^9.0.2",
    "next": "14.2.3",
    "next-auth": "^5.0.0-beta.18",
    "nodemailer": "^6.9.13",
    "react": "^18",
    "react-dom": "^18"
  },
  "devDependencies": {
    "@snaplet/copycat": "^5.0.0",
    "@snaplet/seed": "^0.97.20",
    "@testing-library/jest-dom": "^6.4.6",
    "@testing-library/react": "^16.0.0",
    "@testing-library/user-event": "^14.5.2",
    "@types/jest": "^29.5.12",
    "@types/jsonwebtoken": "^9.0.6",
    "@types/node": "^20",
    "@types/pg": "^8.11.6",
    "@types/react": "^18",
    "@types/react-dom": "^18",
    "eslint": "^8",
    "eslint-config-next": "14.2.3",
    "jest": "^29.7.0",
    "jest-environment-jsdom": "^29.7.0",
    "pg": "^8.12.0",
    "postcss": "^8",
    "tailwindcss": "^3.4.1",
    "ts-node": "^10.9.2",
    "tsx": "^4.15.7",
    "typescript": "^5"
  }

For example, I get this error when using ESM in my test component: Cannot find module '@auth/supabase-adapter' from 'src/components/ui/bar.tsx'

I even get it when mocking it in jest.setup.ts with simply jest.mock('@auth/supabase-adapter'): Cannot find module '@auth/supabase-adapter' from 'jest.setup.ts'

The only solution I found so far is to create the manual __mocks__ folder and put problematic modules there.

Very confusing I think I'm going to give a try to Vitest instead.

rafal-r avatar Jun 28 '24 11:06 rafal-r

@rafal-r Yeah, i stopped using jest and I'm using vitest from now on. less headaches.

leandroruel avatar Jul 04 '24 14:07 leandroruel