workers-sdk
workers-sdk copied to clipboard
cloudflare
Feature Request: Support --node-compat in wrangler pages build
What version of Wrangler are you using?
2.0.6 or beta
What operating system are you using?
Mac
Describe the Bug
App was rewrited from CRA to Remix and i've moved all config from Remix Cloudflare Pages template where it works to my project where i get that error
I've tried to use versions: beta and 2.0.6 doesn't work on both of them.
commands that i've used:
"dev": "remix build && run-p dev:*",
"dev:remix": "remix watch",
"dev:wrangler": "cross-env NODE_ENV=development wrangler pages dev ./public",
Error:
[pages:err] ReferenceError: Buffer is not defined
at /Repos/app-partners-remix/functions/[[path]].js:10:47417
at /Repos/app-partners-remix/functions/[[path]].js:1:822
at /Repos/app-partners-remix/functions/[[path]].js:10:67472
at /Repos/app-partners-remix/functions/[[path]].js:1:822
at /Repos/app-partners-remix/functions/[[path]].js:10:103675
at /Repos/app-partners-remix/functions/[[path]].js:1:822
at /Repos/app-partners-remix/functions/[[path]].js:10:106256
at /Repos/app-partners-remix/functions/[[path]].js:1:822
at /Repos/app-partners-remix/functions/[[path]].js:329:99451
at SourceTextModule.evaluate (node:internal/vm/module:224:23)
✘ [ERROR] Could not start Miniflare.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: "dev:wrangler" exited with 1.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Edit: I found out package @emotion/server/create-instance is responsible for that bug, without it wrangler works.
~The package @emotion/server/create-instance is made for node.js runtimes, which has Buffer. Could you try adding node_compat = true to your wrangler.toml? That may or may not resolve your issue here.~
Disregard that, this is for Pages which doesn't have a node_compat mode yet.
As @threepointone figured out, this might be fixed by passing --node-compat flag, which was released in [email protected] (#1000)
wrangler pages dev --node-compat ./public
This is not a perfect solution since it doesn't work in production just yet, so I wouldn't recommend using it
@threepointone is there any issue tracking we can add for the addition of this node-compat in production for pages?
Hi @nevikashah & @jrf0110, Do you have any updates on the status of this functionality?
I have spent a lot of time converting my codebase to work with pages in node-compat mode but now am at an impasse as it cannot be deployed to the cloud 😕
I try to use --node-compat with wrangler pages publish like described here, but I got same problem :
@jagu-sayan Sharing some of my knowledge here to help save you and others the time that I've spent.
I had some discussions in Discord and was encouraged by a moderator to make this PR: https://github.com/cloudflare/wrangler2/pull/1857
The reviewer rejected it and gave a vague response that they want to "tackle this slightly differently". No idea where node compatibility sits on their roadmap.
You can see some steps here which might work in theory, however did not work the same as in local dev for me: https://discord.com/channels/595317990191398933/910978223968518144/1016371495968182362
I'm just going to continue paying Netlify until a competitor provides a solution that works.
I vaguely remember doing this? @WalshyDev Can I close this out or is it still outstanding?
We have it in Function building and apparently in dev but for some reason not in publish 🤔
We have it in Function building and apparently in
devbut for some reason not inpublish🤔
Probably because I forgot to add it to both 🤣
@JacobMGEvans Someone try to do a PR (https://github.com/cloudflare/wrangler2/pull/1857) and it was rejected.
@JacobMGEvans Someone try to do a PR (#1857) and it was rejected.
@jagu-sayan I am referring to when I implemented it for pages dev I didn't implement it for pages publish. That is interesting that PR got rejected.
Came here hoping there would be an ETA but looks like after almost 12 months nothing. Meanwhile AFAIK this is the only way to get standard OTEL libraries working in CloudFlare.
So we're forced to decide between: 1) ditch OTEL 2) ditch CF Pages.
We recently introduced a new compatibility flag that allows you to use a subset of Node.js APIs that are provided directly by the Workers Runtime, without the need to add polyfills (which is what --node-compat does under the hood).
You can learn more about it here or on in this blog post.
To start using it, add the nodejs_compat compatibility flag to your wrangler.toml:
compatibility_flags = [ "nodejs_compat" ]
That's fantastic news thanks for the update 🎉
How do I enable them in Pages? Couldn't find anything in the links you added, or does it just work or is there a setting in the dashboard?
Yep in the dash! https://dash.cloudflare.com?to=/:account/pages/view/:pages-project/settings/functions
@WalshyDev is there maybe a separate ticket to get these flags into a wrangler.toml for pages too?
@WalshyDev is there maybe a separate ticket to get these flags into a wrangler.toml for pages too?
@RichiCoder1 IIUC $ wrangler pages ... does not read from wrangler.toml in the first (try making it explicit with --config ... flag, it throws an error) place so would seem to invalidate your request. I might be confused.
@RichiCoder1 IIUC $ wrangler pages ... does not read from wrangler.toml in the first (try making it explicit with --config ... flag, it throws an error) place so would seem to invalidate your request. I might be confused.
That's the thrust of my ask. To be able to configure all the various pages options via wrangler.toml, which is not possible today.
@jasonkuhrt — if you're using Pages, you can set compatibility flags in the dashboard or pass the --compatibility-flag to wrangler. Updated docs below:
https://developers.cloudflare.com/workers/runtime-apis/nodejs/
This is still not working for me. When I try wrangler pages dev --compatibility-flag=nodejs_compat --compatibility-date=2023-03-29 --local ./public with code...
import * as Util from 'util I get:
Compiling worker to "/var/folders/kj/42l7g3mx7nn5yvygg5w9t7hh0000gn/T/functionsWorker-0.927152498198341.mjs"...
✘ [ERROR] 1 error(s) and 0 warning(s) when compiling Worker.
✘ [ERROR] Could not resolve "util"
[[path]].js:34539:22:
34539 │ import * as Util from "util";
╵ ~~~~~~
The package "util" wasn't found on the file system but is built into node.
Add "node_compat = true" to your wrangler.toml file to enable Node compatibility.
✘ [ERROR] Build failed with 1 error:
[[path]].js:34539:22: ERROR: Could not resolve "util"
or with import * as Util from "node:util":
[mf:err] VMScriptRunnerError [ERR_MODULE_RULE]: Unable to resolve "yugd9nri6n.js" dependency "node:util": no matching module rules.
If you're trying to import an npm package, you'll need to bundle your Worker first.
at linker (/Users/jasonkuhrt/projects/prisma/pdp-spike-control-plane-v2/node_modules/.pnpm/@[email protected]/node_modules/@miniflare/runner-vm/src/linker.ts:101:13)
at ModuleWrap.<anonymous> (node:internal/vm/module:315:30)
at SourceTextModule.<computed> (node:internal/vm/module:314:36)
at SourceTextModule.link (node:internal/vm/module:199:22)
at VMScriptRunner.runAsModule (/Users/jasonkuhrt/projects/prisma/pdp-spike-control-plane-v2/node_modules/.pnpm/@[email protected]/node_modules/@miniflare/runner-vm/src/index.ts:39:18)
at VMScriptRunner.run (/Users/jasonkuhrt/projects/prisma/pdp-spike-control-plane-v2/node_modules/.pnpm/@[email protected]/node_modules/@miniflare/runner-vm/src/index.ts:86:28)
at EventTarget.#reload (/Users/jasonkuhrt/projects/prisma/pdp-spike-control-plane-v2/node_modules/.pnpm/@[email protected]/node_modules/@miniflare/core/src/index.ts:790:42)
at EventTarget.getPlugins (/Users/jasonkuhrt/projects/prisma/pdp-spike-control-plane-v2/node_modules/.pnpm/@[email protected]/node_modules/@miniflare/core/src/index.ts:1033:5)
at createServer (/Users/jasonkuhrt/projects/prisma/pdp-spike-control-plane-v2/node_modules/.pnpm/@[email protected]/node_modules/@miniflare/http-server/src/index.ts:362:19)
at startServer (/Users/jasonkuhrt/projects/prisma/pdp-spike-control-plane-v2/node_modules/.pnpm/@[email protected]/node_modules/@miniflare/http-server/src/index.ts:469:18)
I am also getting the same errors (e.g. ... dependency "node:util": no matching module rules.) when using wringler dev --local with this config:
name = "pdp-spike-control-plane-v2"
compatibility_date = "2023-03-29"
main = "./build/index.js"
compatibility_flags = ["nodejs_compat"]
[dev]
port = 8788
[site]
bucket = "./public"
@jasonkuhrt — you'll need to use --experimental-local rather than --local. Experimental local mode uses the open source workerd runtime, which has these APIs available, whereas --local mode relies on Miniflare to shim a Workers-like environment.
--experimental-local will become the default (--local) in the near future. For more, see: https://blog.cloudflare.com/miniflare-and-workerd/
We'll update docs to clarify. Thanks for flagging this.
@irvinebroque Thanks going to try that. Am I correct to say that this is not documented anywhere? These details left to discover can eat up a lot of time in an engineers' day. Happy if it's easier for the next developer at least!
--edit ignore my question I just saw now for some reason your text:
We'll update docs to clarify. Thanks for flagging this.
@irvinebroque FYI hitting another issue when I try that https://github.com/cloudflare/workers-sdk/issues/2960.
Another issue. It seems that an import of async_hooks will not work. Instead it must be node:async_hooks. Is that correct?
If so, my next question is, what to do about dependencies that import using "async_hooks" instead of "node:async_hooks"?
Right now I don't know how to work around this issue.
@jasonkuhrt — we're working on ways to make it possible to use nodejs_compat with dependencies that don't import Node APIs from the node: namespace. Stay tuned.
@GregBrimble — think we can close this particular Github issue now that your change has rolled out?