Mark `node:` as external for Node.js compatibility

[chientrm]

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

• [x] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
• [x] This message body should clearly illustrate what problems it solves.
• [x] Ideally, include a test that fails without this PR but passes with it.

Tests

• [x] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

• [x] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. Changesets that add features should be minor and those that fix bugs should be patch. Please prefix changeset messages with feat:, fix:, or chore:.

Explain

Last time I asked to add polyfill for Buffer API. That's no longer neccessary because Cloudflare added a bunch of nodejs APIs to Workers. They are all prefixed by node:.

Example:

import { Buffer } from 'node:buffer';

These APIs are enabled via compatibility_flags nodejs_compat and work for Workers, Pages and wrangler.

The APIs is here: Node.js compatibility

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: 62dd0f5a05b3de17dc4de51a3a18608077957626

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@sveltejs/adapter-cloudflare-workers	Minor
@sveltejs/adapter-cloudflare	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[ghostdevv]

I'm for this change but I think it may make sense having it behind some sort of config option, maybe nodeCompat? That way we can document what this means and not suddenly start allowing node packages for people who don't know what that means on Cloudflare.

[chientrm]

I'm for this change but I think it may make sense having it behind some sort of config option, maybe nodeCompat? That way we can document what this means and not suddenly start allowing node packages for people who don't know what that means on Cloudflare.

I committed the suggested solution yesterday, not sure GitHub will notify the mods so that I quote your reply here.

[ghostdevv]

We should also add notes to the docs about this :pray: We'll also will need some opinions from other maintainers before we can merge

[benmccann]

We generally try to avoid adding options, so unless there's a scenario where we don't want to do it, I would suggest we should always do it.

However, I'm not sure whether it's enough? @cimnine reported in https://github.com/sveltejs/kit/pull/10521 that they had to use alias to actually address an issue using a Node.js module. Is there some kind of guidance we could provide users as to when external is sufficient and when alias would be required?

Update: answering my own question. In https://github.com/sveltejs/kit/pull/10521 the user was trying to use fs. Cloudflare only provides some Node modules and that isn't one of them: https://developers.cloudflare.com/workers/runtime-apis/nodejs/

[ghostdevv]

However, I'm not sure whether it's enough? @cimnine reported in https://github.com/sveltejs/kit/pull/10521 that they had to use alias to actually address an issue using a Node.js module. Is there some kind of guidance we could provide users as to when external is sufficient and when alias would be required?

I'm still not 100% sure what the decision is re: various esbuild options being passable to the adapters - regardless this might be a nice change to reduce friction even if we do end up merging one of the esbuild option related prs - WDYT? @benmccann

[chientrm]

Just in case you want to try out the Node.js compatibility, you can use my forked adapter meanwhile:

@chientrm/adapter-cloudflare

[JustMrMendez]

any updates on this? can't get postgres working on cloudflare and this might be related

[chientrm]

any updates on this? can't get postgres working on cloudflare and this might be related

You can try my adapter while waiting @chientrm/adapter-cloudflare.

Remember to add flag nodejs_compat to your Workers or Pages, and set adapter: adapter({ nodeCompat: true }).

[superboss224]

Hello, will the fix be merged anytime soon ?

[carstenjaksch]

any updates on this? can't get postgres working on cloudflare and this might be related

You can try my adapter while waiting @chientrm/adapter-cloudflare.

Remember to add flag nodejs_compat to your Workers or Pages, and set adapter: adapter({ nodeCompat: true }).

Unfortunately, that doesn't work for me. I get these errors on Cloudflare Pages during build:

The package "stream" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

The same with a lot other Node packages in nodemailer.

[chientrm]

any updates on this? can't get postgres working on cloudflare and this might be related

You can try my adapter while waiting @chientrm/adapter-cloudflare. Remember to add flag nodejs_compat to your Workers or Pages, and set adapter: adapter({ nodeCompat: true }).

Unfortunately, that doesn't work for me. I get these errors on Cloudflare Pages during build:

The package "stream" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

The same with a lot other Node packages in nodemailer.

how do you import stream?

[carstenjaksch]

any updates on this? can't get postgres working on cloudflare and this might be related

You can try my adapter while waiting @chientrm/adapter-cloudflare. Remember to add flag nodejs_compat to your Workers or Pages, and set adapter: adapter({ nodeCompat: true }).

Unfortunately, that doesn't work for me. I get these errors on Cloudflare Pages during build:

The package "stream" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

The same with a lot other Node packages in nodemailer.

how do you import stream?

I do not import it, but nodemailer does.

[chientrm]

you can fork nodemailer and make it import node:stream instead of stream. Docs here: https://developers.cloudflare.com/workers/runtime-apis/nodejs/streams/

[chientrm]

any updates on this? can't get postgres working on cloudflare and this might be related

You can try my adapter while waiting @chientrm/adapter-cloudflare. Remember to add flag nodejs_compat to your Workers or Pages, and set adapter: adapter({ nodeCompat: true }).

Unfortunately, that doesn't work for me. I get these errors on Cloudflare Pages during build:

The package "stream" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

The same with a lot other Node packages in nodemailer.

how do you import stream?

I do not import it, but nodemailer does.

I updated my @chientrm/adapter-cloudflare recently, it enable node compat by default, no need adapter: adapter({ nodeCompat: true }). At this point I don't think the team's gonna merge this pull request. You may aswell fork your own adapter and publish for your personal use.

[benmccann]

Hmm, I don't know how to merge this commit lol.

I merged it via the GitHub UI for you

[mgibbs189]

@chientrm @benmccann @Rich-Harris Thanks! 🎉

[Hebilicious]

@Rich-Harris While things are better, the adapter is still un-usable in some cases :/ For example trying to build a sveltekit project that uses pg.

When bundling a project with wrangler and the node_compat flag (cli or wrangler.toml), several things happens :

• https://github.com/cloudflare/workers-sdk/blob/main/packages/wrangler/src/deployment-bundle/esbuild-plugins/nodejs-compat.ts this plugin flags all node:x modules as externals.

• These polyfills are injected : https://github.com/cloudflare/workers-sdk/blob/ef0642796dbe17b30fd7b83ccfd3efc651ce0a1a/packages/wrangler/src/deployment-bundle/bundle.ts#L334

This allows cloudflare workers and cloudflare pages projects to correctly build if there's a supported library such as "pg" that doesn't use "node:x" specifiers and use incompatible APIs.

Note that there's also an issue with wrangler deploy where this is not possible to specify node_compat: true, therefore the project must be built manually, then deployed.

As a workaround, I'm using a forked version of the cloudflare adapter to inject the polyfills.

Another gotcha is that the nodejs compatibility flag is not compatible with the polyfills while building with wrangler ... But it's still possible to build manually and enable the flag.

I'm not sure what would be a possible solution, but it would be nice to enable esbuild custom settings for the cloudflare adapters.

[g-wozniak]

I am getting exceptions for every package I am trying to use that uses node: in hooks.server.ts. Tried jsonwebtoken and now paseto.

{
  "outcome": "exception",
  "scriptName": "pages-worker--1989843-preview",
  "diagnosticsChannelEvents": [],
  "exceptions": [
    {
      "name": "Error",
      "message": "Dynamic require of \"node:crypto\" is not supported",
      "timestamp": 1708693796319
    }
  ],
}

Using:

• "@sveltejs/kit": "^2.5.0"
• "@sveltejs/adapter-cloudflare": "^4.1.0"

I think, inside 'hooks.server.ts', got a custom handle which performs:

import {V4} from 'paseto'
...
await V4.verify(sessionToken, secretKey)

and I suspect that's the issue.

Got "nodejs_compat" flag enabled, manually set via Cloudflare dashboard but it does not help.

[chientrm]

I am getting exceptions for every package I am trying to use that uses node: in hooks.server.ts. Tried jsonwebtoken and now paseto.

{
  "outcome": "exception",
  "scriptName": "pages-worker--1989843-preview",
  "diagnosticsChannelEvents": [],
  "exceptions": [
    {
      "name": "Error",
      "message": "Dynamic require of \"node:crypto\" is not supported",
      "timestamp": 1708693796319
    }
  ],
}

Using:

• "@sveltejs/kit": "^2.5.0"
• "@sveltejs/adapter-cloudflare": "^4.1.0"

I think, inside 'hooks.server.ts', got a custom handle which performs:

import {V4} from 'paseto'
...
await V4.verify(sessionToken, secretKey)

and I suspect that's the issue.

Got "nodejs_compat" flag enabled, manually set via Cloudflare dashboard but it does not help.

You're sure you have the flag for the preview environment, too? 😉

[g-wozniak]

Yep, I even changed it to something else to see if the build fails and it does. It's worth pointing out that wrangler property is called "node_compat" but the flag itself is "nodejs_compat" which may be confusing for some. So made sure I've got the right one.

My svelte.config.js is as following:

      adapter: adapter({
         routes: {
            include: ['/*'],
            exclude: ['<all>']
         },
      }),

Perhaps something is missing here...

[chientrm]

Yep, I even changed it to something else to see if the build fails and it does. It's worth pointing out that wrangler property is called "node_compat" but the flag itself is "nodejs_compat" which may be confusing for some. So made sure I've got the right one.

Please notice that node_compat and nodejs_compat are two completely different things.

You can ask for support on discord Cloudflare.

[g-wozniak]

According to the documentation the flag name is: "nodejs_compat". Got you though. That means it requires manual deployment via wrangler.

[absktoday]

Is this supposed to work for Pages Adapter or do I have to use Worker adapter, because I am still getting the Could not resolve "crypto" or Could not resolve "node:crypto".