"Module not found" warnings when importing "ably/promises" on nextjs server

[JohnGemstone]

I get the following warnings in my console when importing Ably on server (nextjs).

warn  - ./node_modules/.pnpm/[email protected]/node_modules/keyv/src/index.js
Critical dependency: the request of a dependency is an expression

./node_modules/.pnpm/[email protected]/node_modules/ws/lib/buffer-util.js
Module not found: Can't resolve 'bufferutil' in '/Users/x/Sites/_tests/ably/ably-nextjs-fundamentals-kit/node_modules/.pnpm/[email protected]/node_modules/ws/lib'

./node_modules/.pnpm/[email protected]/node_modules/ws/lib/validation.js
Module not found: Can't resolve 'utf-8-validate' in '/Users/x/Sites/_tests/ably/ably-nextjs-fundamentals-kit/node_modules/.pnpm/[email protected]/node_modules/ws/lib'

These are reproducible by installing the ably-labs next fundamentals example. Navigate to the pub-sub page and publish a message from the server and observe the console.

https://github.com/ably-labs/ably-nextjs-fundamentals-kit/tree/main

This is also reproducible making a nextjs project from scratch using the latest libraries. I experienced them using:

"next": "^13.4.7",
"@ably-labs/react-hooks": "^2.1.1",
"ably": "^1.2.40",

I'm also using pnpm on OSX.

┆Issue is synchronized with this Jira Task by Unito

[sync-by-unito[bot]]

➤ Automation for Jira commented:

The link to the corresponding Jira issue is https://ably.atlassian.net/browse/SDK-3711

[owenpearson]

Hey @JohnGemstone, thanks for reporting this!

I've reproduced the issue using the ably-nextjs-fundamentals-kit project and it appears to only happen when using pnpm, if it's feasible for you, you should be able to just use npm as a workaround for now. We will investigate this and see what we can do to support pnpm out of the box. FWIW it seems the underlying issue is that pnpm is incorrectly trying to use the nodejs version of the library on the client side.

[JohnGemstone]

Nice one thanks, I'll give npm a spin next time. Will leave this open in case you find a workaround 🔎

[sahilprasad]

hey @owenpearson @JohnGemstone, any updates on this? These messages are clogging our dev logs significantly and would love to get rid of them without having to swap to npm.

[evanrooijen]

Samen happens when using turborepo and bun

[Elmosh]

Any update? same issue, also it fails the built when using next --turbo, it would be great if this can be fixed

[Elmosh]

hey @owenpearson , I just migrated to npm just to fix this error, yet the same issue still failing the build.

[VeskeR]

Update:

Use serverComponentsExternalPackages option in next.config.js (read more about the option here):

const nextConfig = {
  // ...
  experimental: {
    serverComponentsExternalPackages: ['ably'],
  },
};

This fixes the warning messages both for npm and pnpm, and the compilation error when using turbopack.

This should also eliminate the need for using overrides field in your package.json for cacheable-request:

"overrides": {
  "cacheable-request": {
    "keyv": "npm:@keyvhq/core@~1.6.6"
  }
}

Read more about this solution in next issue's comment: https://github.com/ably/ably-js/issues/1699#issuecomment-2022702584